1
00:00:00,040 --> 00:00:03,440
We had this like wild discussion
in the office where I was 

2
00:00:03,440 --> 00:00:06,000
talking to one engineer saying, 
what are you using? 

3
00:00:06,000 --> 00:00:07,760
And he's like, oh, I used 
caffeinate. 

4
00:00:08,119 --> 00:00:10,280
And then I think William walked 
in and he's like, oh, I'm on 

5
00:00:10,280 --> 00:00:12,720
amphetamine. 
Like anybody else listening to 

6
00:00:12,720 --> 00:00:14,720
this would be like, the fuck are
these guys talking about? 

7
00:00:15,480 --> 00:00:19,160
But these are like legit apps 
that are there to keep their. 

8
00:00:19,480 --> 00:00:21,360
It's not a bunch of junkies that
are. 

9
00:00:21,720 --> 00:00:28,120
Talking about maybe? 
Where should we start? 

10
00:00:28,480 --> 00:00:32,240
Which one should we do first? 
Why don't we do something with 

11
00:00:32,280 --> 00:00:34,040
you? 
Like I, I think you're pretty 

12
00:00:34,040 --> 00:00:37,280
into the harness stuff, right? 
Let's let's go. 

13
00:00:37,480 --> 00:00:42,400
I I think we can go from like 
the like tech, like the harness,

14
00:00:42,400 --> 00:00:44,520
like how agents are done to the 
human. 

15
00:00:44,800 --> 00:00:47,600
So we'll end on the human. 
Harness to human. 

16
00:00:47,960 --> 00:00:52,120
Honest to human, that's correct.
How much harness is too much 

17
00:00:52,120 --> 00:01:01,160
harness, and what even do you 
feel like is needed on top of 

18
00:01:01,160 --> 00:01:03,680
the harnesses that you get 
out-of-the-box? 

19
00:01:04,200 --> 00:01:06,280
I guess like how would you 
define harness? 

20
00:01:06,560 --> 00:01:08,920
Like how do you see it because 
it's so open? 

21
00:01:09,120 --> 00:01:16,000
Yeah, I think about harness as 
all the stuff that you expose to

22
00:01:16,120 --> 00:01:19,120
the agent to make it perform 
better. 

23
00:01:19,960 --> 00:01:25,600
And so this is I kind of 
recycled the blog post from that

24
00:01:25,600 --> 00:01:31,440
dude Viv from link chain. 
And it's like all of the 

25
00:01:31,960 --> 00:01:36,760
prompts, all of the skills, all 
the if you have like logs, if 

26
00:01:36,760 --> 00:01:42,160
you have the files, file 
systems, the model itself is 

27
00:01:42,160 --> 00:01:45,000
just one thing or the agent 
itself is one piece of it. 

28
00:01:45,200 --> 00:01:50,280
And then you have everything 
that you add on to the LLM that 

29
00:01:50,680 --> 00:01:55,000
helps it perform better. 
You know, Claude code Codex Eli,

30
00:01:55,000 --> 00:01:57,000
these are harnesses in 
themselves, right? 

31
00:01:57,000 --> 00:02:00,160
And then you'll have, you know, 
like I think Viv and Harrison, 

32
00:02:00,160 --> 00:02:01,720
like they've got their own 
harness. 

33
00:02:01,720 --> 00:02:05,000
I think open claw is pie, if I'm
not wrong. 

34
00:02:05,160 --> 00:02:07,720
There's a whole bunch of them in
there and they're I guess like 

35
00:02:07,720 --> 00:02:09,639
different levels, levels of 
abstractions. 

36
00:02:10,080 --> 00:02:13,720
And I'd say like the Claude code
codex Eli is probably the the 

37
00:02:13,720 --> 00:02:15,440
most serious level of 
abstraction, right? 

38
00:02:15,440 --> 00:02:18,360
Like it's the highest. 
Why you you, it's just, you 

39
00:02:18,360 --> 00:02:21,160
know, clearly it's kind of just 
giving in instructions, but then

40
00:02:21,920 --> 00:02:25,280
you can build even more on top 
of that right before you 

41
00:02:25,280 --> 00:02:27,520
actually reach the user. 
So that's why I'm always like 

42
00:02:27,520 --> 00:02:29,600
curious like where that like 
definition is. 

43
00:02:29,600 --> 00:02:34,760
So if we're saying the harness 
is basically everything, are we 

44
00:02:34,760 --> 00:02:36,120
saying that? 
Are we saying it's between like 

45
00:02:36,120 --> 00:02:38,800
the user and like the model, 
like everything in between? 

46
00:02:39,640 --> 00:02:45,920
And then what's So in that case,
I don't know how much it's too 

47
00:02:45,920 --> 00:02:48,920
much like the way I I think 
about it, it's like the job of 

48
00:02:48,920 --> 00:02:51,800
the harness in this case is 
really to do to do things right.

49
00:02:51,800 --> 00:02:59,760
Like agents really like fast 
feedback loops and agents have, 

50
00:03:00,280 --> 00:03:03,320
I guess what you couldn't say is
like this tendency to do really 

51
00:03:03,320 --> 00:03:06,280
dumb things as well. 
So you almost want to protect it

52
00:03:06,280 --> 00:03:11,800
from itself and you wanted to 
have this fast feedback loop so 

53
00:03:11,800 --> 00:03:15,040
when it does do something dumb, 
it can react quickly and adjust,

54
00:03:15,600 --> 00:03:16,200
right? 
I know. 

55
00:03:16,200 --> 00:03:18,160
Like I know what it needs to do 
to adjust. 

56
00:03:18,160 --> 00:03:21,440
And when you say protect it from
itself, you mean protect it from

57
00:03:22,040 --> 00:03:26,680
an agent that is doing the wrong
thing or cause I've also, I was 

58
00:03:26,680 --> 00:03:29,280
literally just having a 
conversation at lunch with my 

59
00:03:29,280 --> 00:03:33,920
friend Mark about protecting 
agents, like sub agents from 

60
00:03:33,920 --> 00:03:37,360
stepping all over each other. 
And so there's many ways you can

61
00:03:37,360 --> 00:03:41,040
protect it from itself, whether 
it's isolation or it's just like

62
00:03:41,040 --> 00:03:43,920
that's true, giving it the right
tool at the right time. 

63
00:03:43,960 --> 00:03:45,000
That's true. 
That's true. 

64
00:03:45,520 --> 00:03:49,760
So like, so it's like a really, 
really simple one. 

65
00:03:50,360 --> 00:03:56,440
I remember in the early days of 
Cleric, we were extremely 

66
00:03:57,040 --> 00:04:01,440
careful about how we wrote tools
and so we didn't give it access.

67
00:04:01,480 --> 00:04:04,000
So you imagine Kubernetes like 
we didn't actually give it 

68
00:04:04,000 --> 00:04:08,960
access to the Cube Cuttle CLI. 
We wrote custom tools that were 

69
00:04:08,960 --> 00:04:12,840
extremely rigid so the agent 
could just do the thing that we 

70
00:04:12,840 --> 00:04:16,640
wanted to do and nothing else. 
But then, you know, models get 

71
00:04:16,640 --> 00:04:20,120
better and then very soon, like,
you know, six months later, that

72
00:04:20,120 --> 00:04:21,600
turns out to be a really dumb 
idea. 

73
00:04:22,040 --> 00:04:24,960
But at the at the time, it makes
a lot of sense, right? 

74
00:04:24,960 --> 00:04:27,920
Because like you're actually 
preventing and doing more dumb 

75
00:04:27,920 --> 00:04:30,680
things. 
But now, like once you know, 

76
00:04:30,800 --> 00:04:34,960
today, these things are just so 
good at composing multiple tools

77
00:04:34,960 --> 00:04:37,680
to get to the answer at once. 
And then you get to a point 

78
00:04:37,680 --> 00:04:40,200
where another harness is like 
the thing that's restricting you

79
00:04:40,200 --> 00:04:42,160
from getting to like, you know, 
better. 

80
00:04:42,200 --> 00:04:45,000
That's why the big question that
I have is how much harness is 

81
00:04:45,000 --> 00:04:50,320
too much harness? 
And like also how much is, if 

82
00:04:50,320 --> 00:04:56,400
you're trying to really map out 
what you want step by step and 

83
00:04:56,400 --> 00:05:00,760
forcing it into one path, 
there's going to be times that 

84
00:05:01,000 --> 00:05:04,240
you want to do that, but then 
there's other times where you 

85
00:05:04,240 --> 00:05:06,840
don't need to. 
And so by doing it, you're kind 

86
00:05:06,840 --> 00:05:08,320
of wasting your time. 
Right. 

87
00:05:08,760 --> 00:05:10,600
Which is why I think it's an 
evolution, right? 

88
00:05:10,600 --> 00:05:14,920
Like, you know, eventually at 
the current state of the model 

89
00:05:15,000 --> 00:05:17,800
and the current state of like 
whatever harness you're using, 

90
00:05:18,960 --> 00:05:22,720
you, whatever you're doing, if 
you're building an agent, I, I 

91
00:05:22,720 --> 00:05:25,680
hope that you're monitoring 
these things and you're having 

92
00:05:25,720 --> 00:05:28,440
these traces to see like why 
it's screwing up and why it's 

93
00:05:28,440 --> 00:05:31,040
not right. 
So, you know, like we have, we 

94
00:05:31,040 --> 00:05:33,320
document failure modes, like 
every time it does an 

95
00:05:33,320 --> 00:05:37,240
investigation, we do a review 
and we see what did it do Well, 

96
00:05:37,640 --> 00:05:42,880
but mostly like we look at, so 
we, we, we, we take the trace 

97
00:05:42,880 --> 00:05:45,640
and we put it through another 
element we have documented like 

98
00:05:45,760 --> 00:05:48,560
labels, right? 
And then we try to figure out 

99
00:05:48,560 --> 00:05:51,160
like, you know, did you spend 
too much time, you know, like, 

100
00:05:51,160 --> 00:05:54,680
for instance, like did we run 
this log query that just flooded

101
00:05:54,680 --> 00:05:57,360
the context window because we, 
we went too wide, right? 

102
00:05:57,680 --> 00:06:01,920
Or we tried a query that, you 
know, took six attempts when it 

103
00:06:01,920 --> 00:06:05,760
should have just taken 1 because
we were just experimenting right

104
00:06:06,360 --> 00:06:10,440
now, the thing that you don't 
want is to be too restrictive. 

105
00:06:10,440 --> 00:06:13,280
But then when you notice that 
it's constantly doing these 

106
00:06:13,280 --> 00:06:16,040
similar mistakes, then that 
becomes more of a design 

107
00:06:16,040 --> 00:06:17,760
question. 
Like is that something that you 

108
00:06:17,760 --> 00:06:21,280
build into the harness so you 
almost protect it from itself, 

109
00:06:21,480 --> 00:06:25,240
like I said, or is this like 
it's OK, right? 

110
00:06:25,240 --> 00:06:28,880
Like it's, it's not, it's not so
fundamental that it's going to 

111
00:06:28,880 --> 00:06:32,360
screw up because if you try to 
overfit and you try to fit fix 

112
00:06:32,360 --> 00:06:35,160
that, you're going to screw up 
something you know elsewhere. 

113
00:06:35,280 --> 00:06:38,560
And like you said, if it gets 
that fast feedback loop and it's

114
00:06:38,560 --> 00:06:41,840
making the mistake but it 
understands it, can it fix it 

115
00:06:41,840 --> 00:06:44,000
and correct course correct in 
flight? 

116
00:06:44,240 --> 00:06:46,000
Exactly. 
So to me, it's almost like a 

117
00:06:46,000 --> 00:06:48,800
game that you're playing where 
the model is just getting better

118
00:06:48,800 --> 00:06:52,120
and better at reasoning and then
it's getting a lot more capable 

119
00:06:52,120 --> 00:06:55,000
at handling more subjective kind
of tasks like this. 

120
00:06:55,200 --> 00:06:57,080
Because obviously like what you 
don't want is to be very 

121
00:06:57,080 --> 00:06:59,200
deterministic where it's like 1 
size fits all. 

122
00:06:59,760 --> 00:07:02,520
Anything beyond like the most 
basic use cases, like I sorry, 

123
00:07:02,520 --> 00:07:04,680
is probably one of the worst 
ones because it's so dynamic, 

124
00:07:04,680 --> 00:07:07,840
it's so complex. 
You actually want that to be 

125
00:07:07,840 --> 00:07:12,520
some play right for the agent. 
Like mistakes are OK, like 100% 

126
00:07:13,000 --> 00:07:16,320
perfect. 
Like investigation is very 

127
00:07:16,320 --> 00:07:18,800
suspicious. 
Like it means that either you 

128
00:07:18,800 --> 00:07:22,560
like, you really overfit to that
like one thing, which is like 

129
00:07:22,560 --> 00:07:25,400
weird, or it was just so simple 
that you probably shouldn't, you

130
00:07:25,400 --> 00:07:27,720
know, pay attention. 
But now if it's like, you know, 

131
00:07:27,720 --> 00:07:30,120
like 20%, it's just not getting 
anywhere. 

132
00:07:30,120 --> 00:07:31,480
It's just constantly getting 
stuck. 

133
00:07:31,880 --> 00:07:34,400
Now you have a problem. 
And then you could then I think 

134
00:07:34,400 --> 00:07:37,960
that makes sense for it to be 
sort of a harness level, you 

135
00:07:37,960 --> 00:07:39,960
know, evaluation. 
And see, maybe the way I would 

136
00:07:39,960 --> 00:07:42,280
frame it is that not everything 
has to be an agent. 

137
00:07:42,760 --> 00:07:46,800
Yeah, right. 
Like maybe an example that I can

138
00:07:46,800 --> 00:07:54,240
think of is, you know, in our in
our CICD pipeline, we go, we we 

139
00:07:54,240 --> 00:07:58,760
have actually like we try to do 
a like a security review and a 

140
00:07:59,440 --> 00:08:01,400
like a code conventions review, 
things like that. 

141
00:08:02,640 --> 00:08:05,120
That does not need like 100 
different tool calls. 

142
00:08:05,120 --> 00:08:07,680
It doesn't need to be like very,
very complex thing. 

143
00:08:08,080 --> 00:08:10,280
It's pretty simple. 
Effectively you're trying to 

144
00:08:10,280 --> 00:08:13,040
see, OK, like what's the DIF 
that's trying to be shipped? 

145
00:08:13,680 --> 00:08:17,160
And then what is some objective 
metric of what good looks like 

146
00:08:17,160 --> 00:08:19,440
that you're passing to this LLM 
flow? 

147
00:08:19,880 --> 00:08:22,120
And then you just want to know 
is it pass or fail? 

148
00:08:22,240 --> 00:08:25,480
And if it's fail, why? 
That sounds pretty deterministic

149
00:08:25,480 --> 00:08:27,880
to me. 
It's got like some LLM involved.

150
00:08:27,960 --> 00:08:30,880
And so maybe that's like, you 
know, so that's where maybe a 

151
00:08:30,880 --> 00:08:33,200
good example of where you don't 
have to overthink things and 

152
00:08:33,200 --> 00:08:36,520
have this really fancy harness 
and really fancy agent probably 

153
00:08:36,520 --> 00:08:39,400
like get something off the shelf
and like it'll work pretty well.

154
00:08:39,960 --> 00:08:42,080
Whereas like, you know, if 
you're trying to do, you know, 

155
00:08:42,080 --> 00:08:45,480
like a coding agent or like an 
SRE agent, yeah, you're not 

156
00:08:45,480 --> 00:08:47,960
going to get away with something
as, you know, simple as that. 

157
00:08:48,720 --> 00:08:53,600
Talk to me more about the 
evolutions that you had to make 

158
00:08:53,680 --> 00:08:56,520
as the models got better within 
Clare. 

159
00:08:57,040 --> 00:09:01,280
Like you were saying, you know 
we we used to do this but now a 

160
00:09:01,440 --> 00:09:05,160
a few months later even we 
realized we don't need to do 

161
00:09:05,160 --> 00:09:08,080
that anymore. 
Yeah, these are always fun when 

162
00:09:08,080 --> 00:09:10,040
you talk about like, all the 
ways you screwed up. 

163
00:09:10,040 --> 00:09:12,640
Yeah, right. 
Instead of talking about like, 

164
00:09:12,680 --> 00:09:14,880
oh, look at, look at, look at. 
I'll say it's done. 

165
00:09:17,240 --> 00:09:21,440
I, I think we're like, you know,
when we started, I remember like

166
00:09:21,440 --> 00:09:24,800
we had this like big debate. 
This is like, you know, late 

167
00:09:24,800 --> 00:09:30,680
2023 and we went really fancy. 
I remember talking about an 

168
00:09:30,680 --> 00:09:34,760
architecture of like an Ant 
colony where, you know, you have

169
00:09:34,760 --> 00:09:39,320
a queen that's talking to its 
workers and its workers are like

170
00:09:39,320 --> 00:09:41,880
very distinct jobs. 
So, you know, in this case, like

171
00:09:41,880 --> 00:09:44,480
the queen would be the incident 
commander almost. 

172
00:09:44,480 --> 00:09:48,600
And like it would just fire off 
all these ants to go get data. 

173
00:09:50,040 --> 00:09:53,720
And I mean, I think we just 
spent a little bit too much time

174
00:09:53,720 --> 00:09:54,600
on that. 
Like it's. 

175
00:09:54,600 --> 00:09:56,160
Trying to create that 
architecture. 

176
00:09:56,280 --> 00:10:00,200
Yeah, because like there was 
clearly a time when, you know, 

177
00:10:00,240 --> 00:10:04,520
just just for context, this was 
GPT 3.54 just came out. 

178
00:10:04,920 --> 00:10:09,120
I don't think Anthropic was 
really seriously on the scene in

179
00:10:09,120 --> 00:10:11,200
the scene at that point. 
Maybe they were, but like 

180
00:10:11,200 --> 00:10:14,720
definitely everybody was like 
really around the open AI models

181
00:10:15,520 --> 00:10:20,480
and they just weren't very good.
I remember I just for fun, I 

182
00:10:20,480 --> 00:10:23,880
looked at our git history and I 
looked at the like the prompts, 

183
00:10:24,520 --> 00:10:26,840
like we were screaming, you 
know, it was like. 

184
00:10:27,160 --> 00:10:30,520
All caps. 
Yeah, like don't do anything but

185
00:10:30,520 --> 00:10:34,280
Jason, like I just please return
Jason like do this, do that. 

186
00:10:34,280 --> 00:10:36,360
It's like all caps exclamation 
marks. 

187
00:10:36,360 --> 00:10:38,960
I don't know if we like never 
expletives, but I wouldn't be 

188
00:10:38,960 --> 00:10:40,960
surprised. 
And then, you know, your 

189
00:10:40,960 --> 00:10:46,120
coupling that kind of kind of, 
you know, basic usage of the 

190
00:10:46,120 --> 00:10:48,760
model and you have such like, 
you know, sort of expensive 

191
00:10:48,760 --> 00:10:50,560
architecture. 
Something has to give, right? 

192
00:10:50,560 --> 00:10:55,240
So then we rebuilt everything 
Like it took us like, I don't 

193
00:10:55,360 --> 00:10:58,760
know, like took us at least like
3 months to guys realize, OK, 

194
00:10:58,760 --> 00:11:00,600
like this is not going to get us
anywhere. 

195
00:11:02,080 --> 00:11:04,800
And then we made it a, we, we 
made it really simple. 

196
00:11:04,800 --> 00:11:08,360
And I think that was good. 
You know, we call it a query 

197
00:11:08,360 --> 00:11:11,760
agent where it was just like, 
OK, like can we just have 

198
00:11:11,760 --> 00:11:15,040
something very reliable where I 
give you a question and you 

199
00:11:15,040 --> 00:11:19,040
figure out where to go and just 
answer the question. 

200
00:11:19,520 --> 00:11:23,440
So it's not like 10 workers, no 
supervisor, blah, blah, blah. 

201
00:11:23,440 --> 00:11:26,680
It's like a single agent. 
And surprising like I think 

202
00:11:26,680 --> 00:11:30,560
that's that's actually like 
cloud code was amazing because 

203
00:11:30,680 --> 00:11:32,720
it was also actually very 
simple. 

204
00:11:33,680 --> 00:11:37,200
And that was, I'm not saying 
like we got simple first, but 

205
00:11:37,520 --> 00:11:40,080
like we went really Uber 
complicated and then we went 

206
00:11:40,080 --> 00:11:41,280
simple. 
And I thought that was like 

207
00:11:41,280 --> 00:11:44,640
really effective. 
The, the other piece was like, 

208
00:11:44,640 --> 00:11:46,880
obviously like the tools thing 
that I talked about, right? 

209
00:11:46,880 --> 00:11:54,320
Like we, it was almost like a 
thing of if you know what you, 

210
00:11:54,440 --> 00:11:59,600
the problems that you want to 
solve, if you fix the tools, we 

211
00:11:59,600 --> 00:12:02,640
felt that that was a good way of
getting this thing to be as 

212
00:12:02,640 --> 00:12:04,920
deterministic as possible where 
we wanted it to be 

213
00:12:04,920 --> 00:12:08,040
deterministic. 
And so then let the, the model 

214
00:12:08,040 --> 00:12:12,160
actually figure out like how to,
you know, compose its way to the

215
00:12:12,160 --> 00:12:15,720
answer. 
But I think, you know, 

216
00:12:15,720 --> 00:12:20,120
eventually like what, you know, 
Clark goes to the world did was 

217
00:12:20,120 --> 00:12:24,480
say, well, actually the models 
have gotten good enough that you

218
00:12:24,480 --> 00:12:28,440
don't have to fix, you know, the
the tools anymore. 

219
00:12:29,120 --> 00:12:32,440
You can just write bash, you can
write Python And run that. 

220
00:12:33,000 --> 00:12:39,680
And So what we had was basically
relaxing that constraint and 

221
00:12:39,680 --> 00:12:43,520
then moving right. 
And this again is like what I'm 

222
00:12:43,520 --> 00:12:48,720
hearing is you're saying, oh, we
just were giving like the 

223
00:12:48,720 --> 00:12:51,920
harness was less deterministic 
of it. 

224
00:12:51,920 --> 00:12:54,400
We weren't so on top of it. 
Exactly. 

225
00:12:55,120 --> 00:12:57,360
But then The thing is, like, you
always pay a cost, right? 

226
00:12:57,360 --> 00:13:01,920
Because when you have tools that
are that deterministic, security

227
00:13:01,920 --> 00:13:06,480
is not a problem because you 
know off the bat exactly what 

228
00:13:06,480 --> 00:13:09,400
the agent can and cannot do. 
Yeah, the moment you give it 

229
00:13:09,400 --> 00:13:11,840
bash in Python now things get 
scary, right? 

230
00:13:11,920 --> 00:13:15,640
And then so we couldn't really 
make that switch until we had to

231
00:13:15,640 --> 00:13:18,880
just sandbox the shit out of 
this agent, right? 

232
00:13:20,480 --> 00:13:23,560
So the way at least like my 
mental model has shifted over 

233
00:13:23,560 --> 00:13:28,440
the years is just you want to 
have as much non determinism 

234
00:13:28,440 --> 00:13:32,200
inside the box, but you want to 
make a really strong box. 

235
00:13:32,480 --> 00:13:34,720
Uh huh. 
So I can't like as much as you 

236
00:13:34,720 --> 00:13:36,080
can. 
I can't you you want. 

237
00:13:36,200 --> 00:13:41,000
To you guys were some of the 
first folks that I heard talking

238
00:13:41,000 --> 00:13:46,400
about sandboxing and you were 
creating environments too, which

239
00:13:46,400 --> 00:13:49,920
is another one that became very 
popular because you had to 

240
00:13:49,960 --> 00:13:55,480
simulate cloud environments to 
see if the SRE agent was running

241
00:13:55,480 --> 00:13:57,400
right. 
Yeah, yeah. 

242
00:13:57,400 --> 00:14:01,560
Like it's, I think probably the 
most complex thing that at least

243
00:14:01,560 --> 00:14:05,520
in our domain, is that every 
environment we would go to is so

244
00:14:05,520 --> 00:14:09,320
different, right. 
Like, I mean, I use myself as an

245
00:14:09,320 --> 00:14:13,280
analogy. 
Yeah, I used to work at Gojek 

246
00:14:13,280 --> 00:14:16,640
running the ML platform and I'm 
working on problems like, you 

247
00:14:16,640 --> 00:14:19,040
know, how do you make model 
inferencing for pricing, driver 

248
00:14:19,040 --> 00:14:22,320
allocation, uh, you know, as as 
efficient as possible. 

249
00:14:22,360 --> 00:14:26,480
It's all in Kubernetes. 
But then everything I just said 

250
00:14:26,760 --> 00:14:30,240
is going to sound very familiar 
to someone at Lyft, at Uber, 

251
00:14:30,240 --> 00:14:33,000
any, any, any kind of ride 
hailing food company. 

252
00:14:33,880 --> 00:14:37,200
But it's not like I can join 
them today and I'm going to be 

253
00:14:37,280 --> 00:14:39,160
productive. 
Yeah, because they've got their 

254
00:14:39,160 --> 00:14:41,680
own way of doing something like,
yeah, the high level problem 

255
00:14:41,680 --> 00:14:45,600
statement is the true is, is, is
the same, but the way they do it

256
00:14:45,600 --> 00:14:47,920
is so different. 
And so, you know, if you assume 

257
00:14:47,920 --> 00:14:50,840
that I am the agent and I've 
gotten really good at these 

258
00:14:50,840 --> 00:14:54,080
tools like Kubernetes and all 
the stuff, I'm still going to 

259
00:14:54,080 --> 00:14:56,520
suck when I move into Uber, 
right? 

260
00:14:57,280 --> 00:15:01,400
And so we, the way we try to 
solve for that, I know, like I, 

261
00:15:01,560 --> 00:15:03,640
I think you had a really good 
job with William and Shreya 

262
00:15:03,640 --> 00:15:06,800
about this, right? 
A few What was it like a year 

263
00:15:06,800 --> 00:15:08,480
ago? 
That was probably a little more 

264
00:15:08,480 --> 00:15:10,280
than a year ago now. 
Right. 

265
00:15:10,840 --> 00:15:12,680
Yeah. 
And so this is the primary 

266
00:15:12,680 --> 00:15:18,800
problem, like how do you test 
very quickly across diverse 

267
00:15:18,800 --> 00:15:22,800
environments and know that your 
agents actually up to the task? 

268
00:15:23,240 --> 00:15:27,640
And so then like in like it's, 
it's been like a really 

269
00:15:27,640 --> 00:15:30,360
interesting phase. 
You know, you're talking about 

270
00:15:30,360 --> 00:15:33,680
like the evolution of cleric. 
The investigation agent has just

271
00:15:33,680 --> 00:15:37,120
become the least important part 
of the stack. 

272
00:15:38,000 --> 00:15:40,680
It's almost like the easiest 
thing to build now, because that

273
00:15:40,680 --> 00:15:42,920
is like not what's actually 
durable. 

274
00:15:43,240 --> 00:15:46,000
It's it's like these 
environments, like how can you 

275
00:15:46,000 --> 00:15:47,800
test and how do you know it's 
working? 

276
00:15:47,800 --> 00:15:50,240
Like that's durable? 
Like the learning stuff, you 

277
00:15:50,240 --> 00:15:55,120
know, we can talk about, it's 
knowing I was like, monitoring 

278
00:15:55,120 --> 00:15:57,000
it and verifying whether it's 
correct. 

279
00:15:57,600 --> 00:16:00,000
This becomes like far more 
harder and more durable 

280
00:16:00,000 --> 00:16:03,280
challenges. 
Whereas I think, you know, you 

281
00:16:03,280 --> 00:16:07,240
can get your cloud code 
connected up to a bunch of MCPS 

282
00:16:07,240 --> 00:16:08,840
and say, hey, go figure out 
what's wrong. 

283
00:16:09,520 --> 00:16:11,160
You'll probably do fine. 
Yeah. 

284
00:16:11,400 --> 00:16:16,800
Because the search problem is 
much less complex than that. 

285
00:16:17,560 --> 00:16:20,080
Is this working like the 
verification problem right? 

286
00:16:20,240 --> 00:16:23,560
Right, right. 
Like once you go from beyond a 

287
00:16:23,560 --> 00:16:27,280
simple kind of task where it's 
like, you know, if something's 

288
00:16:27,280 --> 00:16:30,960
wrong, it's ironic. 
Like the more, the more serious 

289
00:16:30,960 --> 00:16:33,880
the problem. 
It's almost like there are some 

290
00:16:33,880 --> 00:16:36,880
cases of that, but it's actually
very simple because it's 

291
00:16:36,880 --> 00:16:38,360
serious, because something is 
down. 

292
00:16:38,720 --> 00:16:41,960
So it's quite binary, right? 
So it's, it's pretty obvious 

293
00:16:42,000 --> 00:16:45,640
this is the problem, right? 
But like the most subtle it 

294
00:16:45,640 --> 00:16:48,200
gets, now things get hard, 
right? 

295
00:16:48,200 --> 00:16:49,960
Because now it's like very 
reducing credit. 

296
00:16:49,960 --> 00:16:53,600
So that's where it's like really
hard to know. 

297
00:16:53,840 --> 00:16:55,200
That's where the verification 
comes in. 

298
00:16:55,200 --> 00:16:56,240
That's where the learning comes 
in. 

299
00:16:56,240 --> 00:16:59,320
Because like you need to 
actually like look at the 

300
00:16:59,320 --> 00:17:03,280
problem through the lens of that
company, like their conventions,

301
00:17:03,280 --> 00:17:06,400
the way they do things. 
Because if you just come in with

302
00:17:06,400 --> 00:17:09,319
just world knowledge, it's going
to be really complicated. 

303
00:17:09,319 --> 00:17:12,480
Because like the kind that you 
know, out of maybe 100 different

304
00:17:12,480 --> 00:17:16,079
reasons this could have failed, 
you have no way of knowing which

305
00:17:16,079 --> 00:17:19,040
ones you should be paying more 
attention to because they're 

306
00:17:19,040 --> 00:17:21,960
more likely to be the cause. 
Which an engineer who spent ten 

307
00:17:21,960 --> 00:17:24,200
years in the company would 
immediately do, right? 

308
00:17:25,400 --> 00:17:28,960
Which at the end of the day is 
kind of still the search space, 

309
00:17:29,640 --> 00:17:34,320
but you have to verify the 
search in a much different way 

310
00:17:34,320 --> 00:17:36,000
than. 
And I like what you're saying 

311
00:17:36,000 --> 00:17:40,680
there, like the it's a huge 
problem, you know, oh, the 

312
00:17:40,680 --> 00:17:45,640
database blew up or, you know, 
it's normally pretty clear if 

313
00:17:45,640 --> 00:17:49,840
it's a very, very if it's Seb 
one or whatever incident, you're

314
00:17:49,840 --> 00:17:54,320
like we understand. 
But when it's just something 

315
00:17:54,840 --> 00:17:58,920
that you're like, why is latency
spiking by 20% here? 

316
00:17:58,920 --> 00:18:00,680
What is going on? 
Those are the worst. 

317
00:18:00,880 --> 00:18:02,440
Yeah. 
And so then you really have to 

318
00:18:02,440 --> 00:18:07,160
dig in and you have to verify 
through that whole the search 

319
00:18:07,160 --> 00:18:11,200
space again and have the agent 
know if it's on the right path. 

320
00:18:11,720 --> 00:18:13,760
And so that's been where you're 
focusing the attention. 

321
00:18:13,760 --> 00:18:16,920
Now you've and figuring out the 
sandboxes, figuring out the 

322
00:18:16,920 --> 00:18:19,360
environments, figuring out how 
to verify. 

323
00:18:20,000 --> 00:18:25,080
Yeah, like how do you just 
relentlessly and ruthlessly keep

324
00:18:25,080 --> 00:18:29,120
getting the accuracy up, and 
then as you're doing that, you 

325
00:18:29,120 --> 00:18:32,240
keep falling into more and more 
of this subtle territory? 

326
00:18:32,760 --> 00:18:36,200
Then you need to build more 
stuff to make sure that you're 

327
00:18:36,200 --> 00:18:38,400
on the right track, right? 
Because you once you've moved 

328
00:18:38,400 --> 00:18:41,240
away from simple, that's where 
like all this other stuff that 

329
00:18:41,240 --> 00:18:46,040
we talked about matters. 
I think it's almost like that's 

330
00:18:46,040 --> 00:18:50,280
why I've, I've started thinking 
a lot now about just sort of the

331
00:18:50,280 --> 00:18:54,640
human agent interactions now, 
right? 

332
00:18:54,640 --> 00:18:59,760
Because you know, in a world 
where the agents, you know, 

333
00:18:59,760 --> 00:19:02,520
something like 30 to 40% 
correct. 

334
00:19:04,680 --> 00:19:06,880
I actually, I mean, there's a 
little bit of a detour, but 

335
00:19:06,880 --> 00:19:09,280
like, I was, I was talking to 
somebody like an engineer who 

336
00:19:09,280 --> 00:19:13,240
said, I almost like it when the 
agent's always wrong because I'm

337
00:19:13,240 --> 00:19:19,080
suspicious and I'll check right.
Whereas like if it's like mostly

338
00:19:19,080 --> 00:19:22,360
right, I start trusting it. 
And then at one time it's like, 

339
00:19:22,720 --> 00:19:27,360
so, so your Spidey sense is now.
And I'm, I'm, I've just seen the

340
00:19:27,360 --> 00:19:33,040
Spidey sense just continuing to 
get challenged because, you 

341
00:19:33,040 --> 00:19:35,440
know, I can, you know, all the 
stuff I was talking about with 

342
00:19:35,440 --> 00:19:37,880
Tarek in the early days, you 
know, when we're doing these 

343
00:19:37,880 --> 00:19:41,040
tools and things like that, you 
know, what's not that it's like 

344
00:19:41,040 --> 00:19:44,120
what just being explicit about 
is that, you know, we weren't 

345
00:19:44,120 --> 00:19:47,040
very effective outside pretty 
simple kind of problems, right? 

346
00:19:47,040 --> 00:19:50,880
And you didn't trust it at all. 
I imagine exactly like, unless 

347
00:19:50,880 --> 00:19:56,000
it was like very obviously the 
problem and we found it, you, 

348
00:19:56,040 --> 00:19:59,640
you, you, you know, you didn't 
see that trust barrier being. 

349
00:19:59,640 --> 00:20:02,760
So that was like our focus for 
like, you know, it's been like 2

350
00:20:02,760 --> 00:20:05,680
1/2 years of just like trying to
get to that point where like, 

351
00:20:06,040 --> 00:20:08,360
like, you know, you're just 
implicitly trusting this thing 

352
00:20:09,160 --> 00:20:11,440
and that's been happening. 
But now it's creating new 

353
00:20:11,440 --> 00:20:15,720
problems, right? 
Because when that happens, 

354
00:20:17,560 --> 00:20:20,880
people are now trying to throw 
you into more and more complex 

355
00:20:21,000 --> 00:20:23,480
areas, like they want to like, 
give you more work. 

356
00:20:24,800 --> 00:20:28,880
And now it's getting a lot more 
subjective on like what's what's

357
00:20:28,880 --> 00:20:31,240
correct and what's not. 
And then people are worried 

358
00:20:31,240 --> 00:20:36,680
about, well, you know, like, how
am I going to keep learning if 

359
00:20:36,680 --> 00:20:39,240
this thing's getting so much 
better, right. 

360
00:20:39,240 --> 00:20:42,640
And so that's that's that's. 
Where you're saying the human 

361
00:20:42,640 --> 00:20:46,600
aspect comes in, you have folks 
that are starting to be like 

362
00:20:46,760 --> 00:20:51,280
suspicious of it, but also 
they're recognizing that, well, 

363
00:20:51,280 --> 00:20:56,200
if I'm always thrown it at all 
of my problems, then am I 

364
00:20:56,200 --> 00:21:01,120
outsourcing the thinking? 
That, but also, I think it's 

365
00:21:01,120 --> 00:21:04,280
like more of a, this shift that 
people have gone through, which 

366
00:21:04,280 --> 00:21:10,560
is, you know, say, say 12 months
ago, people weren't, I don't 

367
00:21:10,560 --> 00:21:13,640
know if people were actually 
that like concerned, like 

368
00:21:13,640 --> 00:21:17,200
outside people who are like 
really deep into AII don't know 

369
00:21:17,200 --> 00:21:20,600
if engineers thought, but these 
things are really good. 

370
00:21:20,600 --> 00:21:23,440
And I'm not sure. 
I, I, I, I heard that more and 

371
00:21:23,440 --> 00:21:26,520
more. 
No, but we're clearly at a point

372
00:21:26,520 --> 00:21:29,320
like, I think people just went, 
went into Christmas break, 

373
00:21:29,320 --> 00:21:31,560
played around with like all this
stuff. 

374
00:21:31,760 --> 00:21:34,360
And then we're like, holy shit, 
like things have really changed.

375
00:21:35,400 --> 00:21:39,160
And now you can just see this 
trajectory that, you know, 

376
00:21:39,160 --> 00:21:41,800
whatever it's not good at now 
it's going to get there. 

377
00:21:41,840 --> 00:21:44,920
Like I know it is. 
And so now all of us are kind of

378
00:21:44,920 --> 00:21:46,480
kind of thinking like, OK, so 
what's next? 

379
00:21:46,480 --> 00:21:50,280
Like what does what, what 
happens now where I'm not needed

380
00:21:50,280 --> 00:21:55,240
anymore in this one task, right?
So imagine like, you know, six 

381
00:21:55,240 --> 00:21:59,760
months ago you're talking to 
cleric, maybe cleric gets 60% 

382
00:21:59,760 --> 00:22:02,960
there and now you have to like 
have two more follow-ups and get

383
00:22:02,960 --> 00:22:04,520
to their answer and then you're 
done. 

384
00:22:05,720 --> 00:22:08,840
But now he's going 65, now it's 
707580. 

385
00:22:08,840 --> 00:22:11,920
It's just keeping on going. 
And then you're like, OK, So 

386
00:22:11,920 --> 00:22:14,920
what does this mean, right? 
Yeah, where do I add value? 

387
00:22:15,200 --> 00:22:17,560
Yes. 
And that's what I'm really 

388
00:22:17,560 --> 00:22:19,440
interested in. 
And so, so part of it is what 

389
00:22:19,440 --> 00:22:21,200
you said, which is like the 
learning loop. 

390
00:22:22,240 --> 00:22:25,640
But then also I think it's more,
you know, like a more 

391
00:22:25,640 --> 00:22:29,600
philosophical question, right? 
Like where sort of am I So in, 

392
00:22:29,720 --> 00:22:33,880
in my mental model? 
It's like, I don't know, I'm 

393
00:22:33,880 --> 00:22:35,920
still working on like what the 
right analogy would be. 

394
00:22:35,920 --> 00:22:39,760
But it's almost like all of us 
are going to become managers. 

395
00:22:40,880 --> 00:22:46,680
I don't think we're going to 
like, you know, if you look at 

396
00:22:46,680 --> 00:22:49,040
sort of the, the kind of the 
online chatter and like this is 

397
00:22:49,040 --> 00:22:54,080
even happening in our in our 
team, people are token maxing. 

398
00:22:54,800 --> 00:22:58,400
People are like worrying about 
how do I make sure my agents 

399
00:22:58,400 --> 00:23:00,360
have what they need to get work 
done? 

400
00:23:00,840 --> 00:23:02,880
Yeah. 
How does it continue through the

401
00:23:02,880 --> 00:23:05,280
night while I sleep? 
It still is working. 

402
00:23:05,440 --> 00:23:06,640
Right. 
Like I had this, like we had 

403
00:23:06,640 --> 00:23:10,760
this like wild discussion in the
office where I was talking to 

404
00:23:10,800 --> 00:23:12,480
one engineer saying, what are 
you using? 

405
00:23:12,480 --> 00:23:14,200
And he's like, oh, I use 
caffeinate. 

406
00:23:14,600 --> 00:23:16,760
And then I think William walked 
in and he's like, oh, I'm an 

407
00:23:16,760 --> 00:23:19,160
amphetamine. 
Like anybody else listening to 

408
00:23:19,160 --> 00:23:21,200
this would be like, the fuck are
these guys talking about? 

409
00:23:21,920 --> 00:23:25,600
But these are like legit apps 
that are there to keep your. 

410
00:23:25,960 --> 00:23:28,400
It's not a bunch of junkies that
are talking. 

411
00:23:28,880 --> 00:23:34,040
About but but yeah, like so so 
then what? 

412
00:23:34,160 --> 00:23:36,920
You know, if you just like 
double click a little bit beyond

413
00:23:36,920 --> 00:23:41,360
like the sweatshop vibes, it's 
basically it's like, hey, I have

414
00:23:41,360 --> 00:23:46,160
these workers and I kind of care
about making sure that they're 

415
00:23:46,160 --> 00:23:48,040
OK. 
I want to make sure that they're

416
00:23:48,040 --> 00:23:50,360
not going to be interrupted. 
They've got what they need to 

417
00:23:50,360 --> 00:23:53,600
get stuff done. 
It's pretty kind of managerial 

418
00:23:53,600 --> 00:23:55,040
problem. 
Yeah, right. 

419
00:23:55,040 --> 00:23:59,400
It's just the managers don't 
tend to give humans 

420
00:23:59,440 --> 00:24:00,880
amphetamines. 
No. 

421
00:24:00,880 --> 00:24:03,440
And that's. 
Well, not yeah, at least not all

422
00:24:03,440 --> 00:24:05,320
of them. 
Like the ones we you, you should

423
00:24:05,320 --> 00:24:07,280
watch out for. 
Those are the ones. 

424
00:24:07,360 --> 00:24:09,680
That exactly. 
Yeah. 

425
00:24:09,760 --> 00:24:15,480
So, but the apart from that, I 
agree with you wholeheartedly 

426
00:24:15,480 --> 00:24:20,000
because when we had the coding 
agent conference a month back, I

427
00:24:20,000 --> 00:24:26,120
remember one of my friend's 
bosses who is there like VPN 

428
00:24:26,120 --> 00:24:32,680
Jet, a fairly big company. 
Like he was so excited because 

429
00:24:32,680 --> 00:24:38,240
he went to the workshop and he 
was like hands on shipping code,

430
00:24:38,240 --> 00:24:42,720
like merging PRS super excited. 
And then when I wrote about that

431
00:24:42,720 --> 00:24:44,840
on LinkedIn, people were like, 
yeah, well, why wouldn't they 

432
00:24:44,840 --> 00:24:47,800
be? 
Managers have been doing this. 

433
00:24:47,800 --> 00:24:49,680
Like what do you think a manager
is? 

434
00:24:49,680 --> 00:24:53,640
It's almost like a vibe coder. 
You're managing agents or you're

435
00:24:53,640 --> 00:24:56,080
managing people. 
It's just people are actually 

436
00:24:56,080 --> 00:24:58,160
kind of harder to manage a. 
100%. 

437
00:24:58,800 --> 00:25:01,520
Exactly. 
Humans are very messy agents, 

438
00:25:01,520 --> 00:25:04,840
less so. 
You can really grind and give 

439
00:25:04,840 --> 00:25:06,480
them the amphetamines and it's 
all right. 

440
00:25:08,280 --> 00:25:10,080
Yeah. 
Or you can just be. 

441
00:25:10,840 --> 00:25:13,760
Yeah, there's there's no need to
be too nice to your. 

442
00:25:13,760 --> 00:25:14,920
Agents. 
Yeah, polite. 

443
00:25:14,960 --> 00:25:19,520
They don't ask for raises every 
couple months or more, shares 

444
00:25:19,520 --> 00:25:22,400
all of that stuff. 
Or they don't like complain 

445
00:25:22,400 --> 00:25:25,000
about sitting next to somebody 
that smells bad. 

446
00:25:25,040 --> 00:25:29,520
All of that stuff that humans 
have to deal with, right? 

447
00:25:29,720 --> 00:25:34,240
You don't deal with that and. 
But it's funny to me because 

448
00:25:34,360 --> 00:25:38,160
it's, it's like I, I think 
people have gotten really caught

449
00:25:38,160 --> 00:25:39,760
up with how addictive it is, 
right? 

450
00:25:39,760 --> 00:25:44,000
And I think that's coming from 
this place of, you know, like, I

451
00:25:44,000 --> 00:25:49,280
think if you just went through 
and got to, you know, 

452
00:25:49,280 --> 00:25:53,040
eventually, like there instead, 
it's a very stressful job being 

453
00:25:53,040 --> 00:25:56,440
a manager, right? 
Like you're constantly like, 

454
00:25:56,880 --> 00:26:00,440
hey, I've got, you know, 
10/20/30 people depending on me 

455
00:26:01,120 --> 00:26:03,080
and what I say we're going to 
do. 

456
00:26:04,200 --> 00:26:09,320
I've got 30 times like the 
bandwidth of that week assigned 

457
00:26:09,320 --> 00:26:11,400
to this problem that I've said 
we have to do. 

458
00:26:11,800 --> 00:26:15,080
And now you know, like, and this
problem goes from CEO to 

459
00:26:15,080 --> 00:26:18,400
engineering manager. 
It's pretty freaking worrying, 

460
00:26:18,400 --> 00:26:19,160
right? 
Like you. 

461
00:26:19,160 --> 00:26:22,240
And so like what I'm seeing with
the amphetamine and like all 

462
00:26:22,240 --> 00:26:26,440
these conversations is like the 
the Colonel is that same worry 

463
00:26:26,440 --> 00:26:30,520
like I I've got all this like 
bandwidth of tokens. 

464
00:26:31,120 --> 00:26:34,440
Am I spending it correctly? 
Like am I doing the right thing?

465
00:26:34,760 --> 00:26:37,640
Some of us aren't, but just like
shipping all kinds of shit, but 

466
00:26:37,640 --> 00:26:40,440
like generally like it's it's 
just going to end up there, 

467
00:26:40,440 --> 00:26:42,440
right. 
And and then you're back to 

468
00:26:42,440 --> 00:26:46,200
being this like person who's 
like, OK, like here's I've got X

469
00:26:46,200 --> 00:26:49,240
resources. 
I've got a problem to sort out. 

470
00:26:49,640 --> 00:26:54,760
How do I, you know, corral my 
army of people or like agents to

471
00:26:54,760 --> 00:26:56,480
actually go, you know, solve 
this problem? 

472
00:26:56,520 --> 00:26:59,080
Yeah, make sure we're all rowing
in the right direction and 

473
00:26:59,320 --> 00:27:02,800
taking advantage of every moment
because any moment that the 

474
00:27:02,800 --> 00:27:05,560
agents aren't right, looping 
well. 

475
00:27:05,560 --> 00:27:07,880
Hopefully we don't get too 
stressed out about that because 

476
00:27:07,880 --> 00:27:09,840
that, that is a like a dark 
hole. 

477
00:27:09,920 --> 00:27:12,600
Yeah, I think. 
But yeah, exactly. 

478
00:27:12,600 --> 00:27:16,040
You don't have 8 agents 
constantly running and you feel 

479
00:27:16,040 --> 00:27:18,000
like you're what are you even 
doing right? 

480
00:27:18,000 --> 00:27:21,120
I know. 
I mean, I feel that like I, I, 

481
00:27:21,120 --> 00:27:24,600
I've, I don't know the last time
why I haven't. 

482
00:27:24,960 --> 00:27:27,480
So, so we use, I'm sure like 
people are using their own 

483
00:27:27,480 --> 00:27:30,440
terms, but like internally we 
use the term like the hopper. 

484
00:27:30,440 --> 00:27:32,080
It's like you're cooking like, 
well, I'm just going to put 

485
00:27:32,080 --> 00:27:35,880
another one on the hopper. 
And so I, I, you know, I, I 

486
00:27:35,880 --> 00:27:39,400
need, I feel like I have this 
need to put something on the 

487
00:27:39,400 --> 00:27:43,720
hopper before I go to bed. 
And I have just infinite 

488
00:27:43,720 --> 00:27:46,080
satisfaction waking up to see 
it's like almost done. 

489
00:27:46,080 --> 00:27:48,800
And they're pretty cool with, 
with the harness discussion and 

490
00:27:48,800 --> 00:27:51,800
all that stuff as well, because 
like you're what you're 

491
00:27:51,800 --> 00:27:54,360
implicitly doing is every time 
it screws up, you're kind of 

492
00:27:54,360 --> 00:27:56,440
thinking, oh, what should I have
given it? 

493
00:27:56,600 --> 00:27:57,960
Like, what could I have given 
this? 

494
00:27:57,960 --> 00:28:00,480
Like, you know, and it's like 
really what managers do, right? 

495
00:28:00,480 --> 00:28:03,520
Like, you know, I was, I was 
reading a post from, I can't 

496
00:28:03,520 --> 00:28:06,120
remember who, but she was like 
the first thing she asked on her

497
00:28:06,120 --> 00:28:10,600
one-on-one was like, well, how, 
what's your work set up like? 

498
00:28:10,720 --> 00:28:12,600
And can I fix something like, 
are you comfortable? 

499
00:28:13,160 --> 00:28:15,280
I mean, are we literally doing 
that with agents now? 

500
00:28:16,000 --> 00:28:18,480
That's the harness. 
That is the harness exactly. 

501
00:28:20,040 --> 00:28:22,320
What I've been seeing, and I 
imagine this is going to be 

502
00:28:22,320 --> 00:28:26,080
something that maybe it's 
fleeting, maybe it becomes more 

503
00:28:26,080 --> 00:28:31,880
prevalent, is that you have to 
do a lot of the upfront work in 

504
00:28:31,880 --> 00:28:35,280
order to be able to put 
something on the hopper and let 

505
00:28:35,280 --> 00:28:39,320
it run. 
Yes, yes, but by a print work I 

506
00:28:39,320 --> 00:28:42,160
assume you mean like, you know, 
speaking it out properly. 

507
00:28:42,160 --> 00:28:45,280
The scaffolding around it, you 
can't just kind of like Willy 

508
00:28:45,280 --> 00:28:48,840
nilly throw in one or two 
sentences and be like go and 

509
00:28:48,840 --> 00:28:51,080
have fun. 
Well, you can, but then you're 

510
00:28:51,080 --> 00:28:53,000
just going to have a look at 
pretty shitty. 

511
00:28:53,400 --> 00:28:54,880
Shitty outcome. 
Exactly. 

512
00:28:54,920 --> 00:28:57,320
I mean, that's where I guess, 
like cost and things like that 

513
00:28:57,320 --> 00:29:02,240
come in. 
But yeah, So what I'm thinking 

514
00:29:02,240 --> 00:29:05,720
is like, OK, so right now we're 
kind of like, you know, some of 

515
00:29:05,720 --> 00:29:08,760
us more than others are using 
this more and more like 

516
00:29:08,760 --> 00:29:11,960
effectively. 
And then, you know, what are 

517
00:29:11,960 --> 00:29:14,120
these people doing that's more 
effective and how are we 

518
00:29:14,120 --> 00:29:15,560
learning from them? 
I'm like, where does this all 

519
00:29:15,560 --> 00:29:18,320
go? 
Like to your point, you know, 

520
00:29:18,320 --> 00:29:20,920
this whole spectrum and 
development and the Ralph Wigan 

521
00:29:20,920 --> 00:29:23,120
loop and all the stuff like it's
coming up because like, you 

522
00:29:23,160 --> 00:29:27,040
know, these are like decks was 
amazing at the coding engine as 

523
00:29:27,040 --> 00:29:29,880
you always like. 
It's all like these lessons 

524
00:29:29,880 --> 00:29:31,960
learned from people who are just
like doing it all the time. 

525
00:29:31,960 --> 00:29:33,160
And we're all trying to learn 
from it. 

526
00:29:34,360 --> 00:29:38,560
But I guess like my, my, my 
thought here is that I think 

527
00:29:39,320 --> 00:29:43,600
it's kind of like you want to be
a mix of a manager and a 

528
00:29:43,600 --> 00:29:47,640
Craftsman, right? 
Because, you know, in a world 

529
00:29:47,640 --> 00:29:53,800
where you can build anything you
want effectively, what is left 

530
00:29:53,880 --> 00:29:56,560
is your taste and your 
judgement, right? 

531
00:29:56,560 --> 00:30:00,600
And that's why I think there's a
very dark way of taking this, 

532
00:30:00,600 --> 00:30:02,400
which is that you lose all 
agency. 

533
00:30:02,400 --> 00:30:06,280
You stop learning like you like 
this thing is just gonna and I, 

534
00:30:06,720 --> 00:30:08,760
I don't know, I guess like if 
you take any anything from this 

535
00:30:08,760 --> 00:30:11,480
conversation, I hope that we can
have a more positive view of it,

536
00:30:11,480 --> 00:30:14,840
which is that like actually the 
really fun stuff about building 

537
00:30:14,840 --> 00:30:17,280
product is putting your 
personality and like your 

538
00:30:17,280 --> 00:30:19,600
thoughts and like your view of 
like what you want to bring to 

539
00:30:19,600 --> 00:30:21,560
the world out there. 
Now you can get there much 

540
00:30:21,560 --> 00:30:25,320
faster. 
And so like, how do we build for

541
00:30:25,320 --> 00:30:31,560
a world like that where that 
becomes like all the why you 

542
00:30:31,560 --> 00:30:34,400
spend all your time on, whereas 
like the sort of the toil of 

543
00:30:34,400 --> 00:30:36,520
getting there is something with 
the agent is doing. 

544
00:30:38,120 --> 00:30:40,960
It goes back to those feedback 
loops you were saying before. 

545
00:30:41,080 --> 00:30:44,120
That's what I was thinking. 
Like if you know, if you, 

546
00:30:44,320 --> 00:30:46,960
because the thing with I think 
the managerial word, which I 

547
00:30:46,960 --> 00:30:52,440
don't like as much is that it 
almost suggests that you don't 

548
00:30:52,440 --> 00:30:56,400
know much about the craft. 
You're just focusing on the sort

549
00:30:56,400 --> 00:30:59,440
of public mastering aspect of 
it, which I mean, I, that's why 

550
00:30:59,440 --> 00:31:02,080
I don't like the word. 
I like the whole craftsmanship 

551
00:31:02,080 --> 00:31:05,200
thing because like it, it, it's 
about mastery, right? 

552
00:31:05,200 --> 00:31:07,400
So you're almost like in charge 
of a workshop and you're trying 

553
00:31:07,400 --> 00:31:10,600
to get something done. 
And what you don't want is to 

554
00:31:10,600 --> 00:31:14,960
judge based on what's produced. 
You actually want to know what's

555
00:31:14,960 --> 00:31:18,320
going on, right? 
You, you don't want to be 

556
00:31:18,320 --> 00:31:21,080
abstracted away too much. 
So I think like, even with the 

557
00:31:21,080 --> 00:31:25,120
agents, I think the mistake 
would be to say this thing's so 

558
00:31:25,120 --> 00:31:27,200
good. 
So we're just going to wait for 

559
00:31:27,200 --> 00:31:29,440
it to finish all of its code and
then I'm going to review all of 

560
00:31:29,440 --> 00:31:31,560
its code. 
I'm not sure that that's 

561
00:31:31,720 --> 00:31:33,840
effective. 
Well, you didn't want to say 

562
00:31:33,840 --> 00:31:38,320
something about and before we 
hit record about how you want to

563
00:31:38,320 --> 00:31:42,600
be able to see as it's making 
its decisions, what it's making 

564
00:31:42,600 --> 00:31:46,040
and then steer it. 
And that's steerability I think 

565
00:31:46,040 --> 00:31:49,560
is a little bit lacking 
sometimes. 

566
00:31:50,280 --> 00:31:55,320
But also, if you're going to try
and scale to running 8 parallel 

567
00:31:55,320 --> 00:32:00,520
agents, you just can't expect to
steer 8 agents in parallel. 

568
00:32:01,480 --> 00:32:03,720
That's true. 
But that's why I think the point

569
00:32:03,720 --> 00:32:05,360
of abstraction is like useful, 
right? 

570
00:32:05,360 --> 00:32:09,040
Because you know, if you've got 
8 agents and you're writing 

571
00:32:09,040 --> 00:32:13,720
some, you know, big project, 
it's going to produce just reams

572
00:32:13,720 --> 00:32:16,640
and reams of code. 
You can't read all that code. 

573
00:32:17,760 --> 00:32:20,920
But an abstraction above that is
what are the decisions that it 

574
00:32:20,920 --> 00:32:24,880
took to get to that code, right?
Like a really dumb example is, 

575
00:32:26,000 --> 00:32:29,000
you know, an agent is probably 
gonna be really good at writing 

576
00:32:29,040 --> 00:32:32,840
a Mac OS desktop app. 
It's gonna get, like, really 

577
00:32:32,840 --> 00:32:34,240
good. 
It's gonna look beautiful. 

578
00:32:34,920 --> 00:32:39,320
But maybe the judgement is that 
we shouldn't write a desktop 

579
00:32:39,320 --> 00:32:41,760
app. 
It should be a web app, right? 

580
00:32:41,760 --> 00:32:45,280
Because I want more customers. 
And like, I, I think there's a 

581
00:32:45,280 --> 00:32:47,680
bunch of people in Linux who's 
gonna like this stuff as well. 

582
00:32:48,960 --> 00:32:51,680
I mean, that's a judgement call.
And so then there's a deletion 

583
00:32:51,680 --> 00:32:56,320
trace somewhere where your agent
or your product manager agent is

584
00:32:56,320 --> 00:32:59,920
like, we should build a Mac App.
And so then, you know, at that 

585
00:32:59,920 --> 00:33:03,400
point you could probably go, 
well, no, we shouldn't. 

586
00:33:03,960 --> 00:33:07,240
I think we should build X 
instead, so that is, I think, 

587
00:33:07,240 --> 00:33:12,400
infinitely easier to review than
all the code that was written, 

588
00:33:12,680 --> 00:33:15,040
right? 
But isn't that just making you 

589
00:33:15,040 --> 00:33:18,080
the bottleneck? 
Well, that's the thing, like I 

590
00:33:18,080 --> 00:33:20,920
think it if you become the 
bottleneck, then it should be 

591
00:33:20,920 --> 00:33:23,760
for a good reason. 
You are actually like putting, 

592
00:33:23,760 --> 00:33:25,920
inserting yourself in there. 
I think there's the kind of two 

593
00:33:25,920 --> 00:33:29,480
constraints like 1 is your test.
So hopefully, like, you know, 

594
00:33:29,480 --> 00:33:32,240
you are operating at that level 
where you're injecting a test 

595
00:33:32,240 --> 00:33:34,520
and that ideally is like 
something you're exercising and 

596
00:33:34,520 --> 00:33:37,200
you're getting better at. 
And the second one is like, I 

597
00:33:37,200 --> 00:33:39,800
think the real world also has 
constraints, right? 

598
00:33:39,800 --> 00:33:43,000
Like in a world where everything
gets amazing and like you can 

599
00:33:43,000 --> 00:33:47,360
produce infinite code, your 
customers can't take that much 

600
00:33:47,360 --> 00:33:52,040
change anyway, right? 
Like you can't have, you know, 

601
00:33:52,080 --> 00:33:54,240
like a lot of these companies 
and the services that we are 

602
00:33:54,240 --> 00:33:57,600
used to took decades to evolve 
into what they are. 

603
00:33:57,680 --> 00:34:01,440
Like, you know, if I, if you 
just speed rang someone who uses

604
00:34:01,520 --> 00:34:05,240
Uber today to everything that's 
they've offered, you know, over 

605
00:34:05,240 --> 00:34:08,520
the last 10-15 years, I mean, 
that's not possible that people 

606
00:34:08,520 --> 00:34:11,560
can't take it. 
So I think, yeah, it is a 

607
00:34:11,560 --> 00:34:13,480
bottleneck. 
But like it's they only make 

608
00:34:13,480 --> 00:34:15,679
sense if it's adding value. 
And I think there's those are 

609
00:34:15,679 --> 00:34:18,480
two ways of adding value. 
Well, that's a fascinating topic

610
00:34:18,480 --> 00:34:24,600
to think about. 
Is where you need the human to 

611
00:34:24,600 --> 00:34:28,360
plug in. 
Because if it is, then going 

612
00:34:28,360 --> 00:34:35,360
back to your example of the 
desktop app versus a web app, 

613
00:34:36,480 --> 00:34:40,159
that's one question that 
potentially you plug in a human 

614
00:34:40,159 --> 00:34:44,920
for or you don't, and it's 
because maybe it just doesn't 

615
00:34:44,920 --> 00:34:50,639
matter and you can build it. 
But like, what decisions need to

616
00:34:50,679 --> 00:34:53,679
actually be outsourced to the 
human? 

617
00:34:54,920 --> 00:34:56,719
You know, where do you plug the 
human in? 

618
00:34:56,719 --> 00:35:00,840
And how does the model know that
this is a really big decision? 

619
00:35:01,200 --> 00:35:05,480
Before I make this decision and 
go down this potential path, I 

620
00:35:05,480 --> 00:35:12,160
should probably ask someone. 
Yeah, I, I mean, I think it's, 

621
00:35:12,320 --> 00:35:14,760
it's useful thinking about like 
what the ends did. 

622
00:35:14,760 --> 00:35:17,480
And I mean, there's no true end 
state, but at least like I think

623
00:35:17,480 --> 00:35:20,160
in the next 12 months, like what
where I think we should be. 

624
00:35:20,880 --> 00:35:24,520
It feels to me like you're kind 
of approving exceptions, right? 

625
00:35:24,520 --> 00:35:27,840
Like when you know, like say you
have an employee and they're 

626
00:35:27,840 --> 00:35:30,400
supposed to like, you know, do 
something like, for instance, 

627
00:35:30,400 --> 00:35:32,440
like book this podcast studio, 
right? 

628
00:35:33,080 --> 00:35:35,880
It's, it's, it's likely that 
you're giving them some 

629
00:35:35,880 --> 00:35:37,640
guidelines, right? 
Like here's your budget. 

630
00:35:37,800 --> 00:35:41,160
Here's kind of the area that I 
want you to look at and you 

631
00:35:41,160 --> 00:35:44,160
know, you go figure it out. 
But if you're going to violate 

632
00:35:44,160 --> 00:35:47,160
any of these bounds, and there's
a good reason, then I need to 

633
00:35:47,160 --> 00:35:50,200
know and I I think it's kind of 
similar here. 

634
00:35:50,680 --> 00:35:54,040
But again, then you have to 
really upfront define those 

635
00:35:54,040 --> 00:35:58,400
bounds clearly and put that 
scaffolding in place and then 

636
00:35:58,400 --> 00:36:01,280
let it run. 
But but I think that's the work 

637
00:36:01,280 --> 00:36:03,200
which is valuable. 
Yeah, right. 

638
00:36:03,200 --> 00:36:06,920
Because I think those are the 
decisions which today, like, you

639
00:36:06,920 --> 00:36:09,680
know, live in people's heads is 
all implicit decision making 

640
00:36:09,680 --> 00:36:12,560
that's happening, right? 
Which is not explicit, but it, 

641
00:36:12,600 --> 00:36:15,400
it was OK. 
Or at least it's OK in a world 

642
00:36:15,400 --> 00:36:18,360
of humans. 
Because when we are, when we're 

643
00:36:18,360 --> 00:36:21,640
not sure, you call a meeting, 
you send a slack or things like 

644
00:36:21,640 --> 00:36:25,560
that, but it just does not work 
with agents because agents work 

645
00:36:25,560 --> 00:36:27,800
on agent time, right? 
Like you, it needs an answer 

646
00:36:27,800 --> 00:36:30,240
right now. 
Like from an SRE standpoint, 

647
00:36:30,240 --> 00:36:32,280
like, well, you know, that's a 
domain, you know, I think about 

648
00:36:32,280 --> 00:36:36,240
a lot. 
It's like how can you reliably 

649
00:36:36,240 --> 00:36:39,400
get an agent to fix something at
2:00 AM if it doesn't have clear

650
00:36:39,400 --> 00:36:41,640
bounds of what it can do and 
what it can't? 

651
00:36:41,640 --> 00:36:43,720
It just can't. 
It can't call a meeting at 2:00 

652
00:36:43,720 --> 00:36:47,160
AM like we call meetings at 2:00
AM because it's an incident and 

653
00:36:47,160 --> 00:36:49,160
like you call everybody and was 
like, guys, what do we do? 

654
00:36:49,160 --> 00:36:50,800
Like should we restart this? 
Should we stop it? 

655
00:36:50,800 --> 00:36:51,920
Should we do this? 
Should we do that? 

656
00:36:53,000 --> 00:36:57,600
Unless those decisions become, 
you know, preserved or like, you

657
00:36:57,600 --> 00:37:00,400
know, queryable, you're not 
going to get anywhere. 

658
00:37:00,400 --> 00:37:05,400
So I think the phase where in 
now it's all about actually, you

659
00:37:05,400 --> 00:37:08,040
know, I think, you know, Jaya 
from Foundation Capital did an 

660
00:37:08,040 --> 00:37:11,000
amazing piece called Context 
graphs and which really just 

661
00:37:11,000 --> 00:37:13,160
blew up. 
But I think it's like all around

662
00:37:13,160 --> 00:37:16,280
kind of a similar idea here 
where that's the valuable data 

663
00:37:16,280 --> 00:37:18,440
now. 
Like what are the decisions that

664
00:37:18,440 --> 00:37:20,560
you know, are OK? 
Like what are the bounds? 

665
00:37:20,560 --> 00:37:22,680
And then how do you actually 
track the decisions that the 

666
00:37:22,680 --> 00:37:26,200
agents are making so that you're
actually having this compounding

667
00:37:26,280 --> 00:37:30,800
loop of preserving? 
And it's these instructions in a

668
00:37:30,800 --> 00:37:33,920
way that humans can read them 
and, and like, go, yeah, yeah, 

669
00:37:33,920 --> 00:37:35,680
that looks fine. 
No, no, don't do this. 

670
00:37:35,880 --> 00:37:37,720
Or at least you're defining an 
exception. 

671
00:37:37,720 --> 00:37:40,160
Like, as long as you're over 
here, do whatever you want. 

672
00:37:40,160 --> 00:37:43,000
But if you're coming out, talk 
to me because I need to tell you

673
00:37:43,000 --> 00:37:45,640
what to do. 
Are you guys doing that? 

674
00:37:45,640 --> 00:37:48,320
Is that one of those more 
complex things that you started 

675
00:37:48,320 --> 00:37:53,800
to add where you have decisions 
being made in Slack and you need

676
00:37:53,800 --> 00:37:57,440
to then incorporate that into 
the agent so that if it 

677
00:37:57,440 --> 00:37:59,800
encounters this problem again in
the future, it kind of has a 

678
00:37:59,800 --> 00:38:02,000
road map? 
We're doing that. 

679
00:38:02,000 --> 00:38:06,040
So that to me is like part of 
learning where you I don't think

680
00:38:06,040 --> 00:38:10,720
it's as advanced as I'd like, 
but that's the path which I want

681
00:38:10,720 --> 00:38:13,000
to be on. 
So you know, I've been railing 

682
00:38:13,000 --> 00:38:16,400
on, you know, on social media 
about like this for deployed 

683
00:38:16,400 --> 00:38:19,840
engineer kind of model. 
The reason I'm not bullish on 

684
00:38:19,840 --> 00:38:23,840
that is that you're actually not
building the thing I'm talking 

685
00:38:23,840 --> 00:38:26,520
about, right? 
Like what I want when you deploy

686
00:38:26,520 --> 00:38:30,800
cleric is I'm giving it every 
opportunity to learn these 

687
00:38:30,800 --> 00:38:33,800
rules. 
And if you don't give it that 

688
00:38:33,800 --> 00:38:35,920
opportunity and you're just 
going to throw engineers at the 

689
00:38:35,920 --> 00:38:39,880
problem and try to like tune it,
then you're not solving that 

690
00:38:39,880 --> 00:38:44,000
real valuable problem, right? 
So to your point on Slack, 

691
00:38:44,080 --> 00:38:46,320
there's a wealth of information,
like when we get added to 

692
00:38:46,320 --> 00:38:50,400
another channel, I mean, there's
a ton of stuff like, you know, 

693
00:38:50,400 --> 00:38:54,080
when this thing goes, goes down,
usually it's because of this 

694
00:38:54,080 --> 00:38:57,040
other thing. 
And here's what, you know, over 

695
00:38:57,040 --> 00:38:59,880
the last six months, our 
engineers have tried and like 

696
00:38:59,880 --> 00:39:02,640
this is what generally works. 
This is what doesn't you can 

697
00:39:02,640 --> 00:39:04,840
build a pretty effective map 
just from this data. 

698
00:39:04,840 --> 00:39:07,200
It's, you know, it's like 
unstructured data. 

699
00:39:07,280 --> 00:39:09,680
It's like it's, it's, it's very 
valuable, right? 

700
00:39:09,680 --> 00:39:13,760
And then that can be replayed 
back to the human to say, here's

701
00:39:13,760 --> 00:39:16,240
what I learned. 
This is look right, right. 

702
00:39:16,320 --> 00:39:20,880
And then now to a human that's 
actually like a very compressed 

703
00:39:20,880 --> 00:39:24,920
form of knowledge, like these 
are sort of what I've learned, 

704
00:39:24,920 --> 00:39:27,200
like what I should do, how your 
environment works. 

705
00:39:27,200 --> 00:39:29,280
Does it look right? 
And to them it's like very easy 

706
00:39:29,280 --> 00:39:32,720
to just edit it. 
Yeah, I do like the distinction 

707
00:39:32,720 --> 00:39:36,600
too of how you're saying the 
real value is in that type of 

708
00:39:36,600 --> 00:39:43,560
thing and being able to create 
these paper trails in a way or 

709
00:39:43,560 --> 00:39:50,880
create that like decision making
process the Sops and then doing 

710
00:39:50,880 --> 00:39:55,440
the upfront work of creating 
those so that the agent can do 

711
00:39:55,440 --> 00:39:58,560
the hard work of like actually 
implementing it, Yes. 

712
00:39:59,800 --> 00:40:02,600
Exactly. 
And you, you want to design your

713
00:40:02,600 --> 00:40:08,200
systems to make that hard work 
like valued, right? 

714
00:40:08,200 --> 00:40:11,400
Like, like it, it has to happen 
like we've tried a lot of times 

715
00:40:11,400 --> 00:40:13,440
like to do like in the SRO 
world. 

716
00:40:14,080 --> 00:40:17,600
You'd insist on teams doing 
postmortems, documenting them, 

717
00:40:17,640 --> 00:40:21,560
things like that. 
It's really hard because you're 

718
00:40:21,560 --> 00:40:24,160
asking people to do something 
with no immediate tangible 

719
00:40:24,160 --> 00:40:26,640
value. 
Like unless you've really got a 

720
00:40:26,640 --> 00:40:28,680
good process down. 
And this happens in every 

721
00:40:28,680 --> 00:40:30,600
industry, right? 
Like just document your shit. 

722
00:40:31,080 --> 00:40:32,760
Like, just make sure you're 
writing stuff down. 

723
00:40:32,760 --> 00:40:36,280
Everybody knows that it's a good
idea, but it's not necessarily. 

724
00:40:37,400 --> 00:40:40,160
But I think agents are super 
interesting because now you do 

725
00:40:40,160 --> 00:40:42,640
get that feedback very quickly, 
right? 

726
00:40:42,640 --> 00:40:46,760
Like if I write something down, 
oh, I tell the agent, hey, 

727
00:40:46,760 --> 00:40:49,640
remember this. 
And then the next time something

728
00:40:49,640 --> 00:40:52,320
happens, it's like, oh, yeah, 
thanks, Dee, you told me about 

729
00:40:52,320 --> 00:40:53,920
this. 
So I, you know, I did it this 

730
00:40:53,920 --> 00:40:56,760
way instead. 
Now you feel, oh, OK, that's 

731
00:40:56,760 --> 00:41:01,040
that's cool. 
Right, like so in a way, what 

732
00:41:01,040 --> 00:41:05,600
you're constantly doing is 
updating documentation and 

733
00:41:05,600 --> 00:41:07,880
updating. 
Pretty much like just for the 

734
00:41:07,920 --> 00:41:10,160
agent. 
Yeah, just for the agent and, 

735
00:41:10,400 --> 00:41:13,560
and I remember, I think William 
told me, yeah, normally you 

736
00:41:13,560 --> 00:41:18,760
don't necessarily need to ingest
the whole Slack thread because 

737
00:41:18,760 --> 00:41:21,680
by the end of a Slack thread, 
there's a decision that's made. 

738
00:41:22,360 --> 00:41:27,280
And did you do it with the is it
the user experience that like at

739
00:41:27,280 --> 00:41:29,880
the end of the Slack thread, 
once you know what needs to 

740
00:41:29,880 --> 00:41:34,080
happen, then you just like at 
Cleric and you know, like update

741
00:41:34,080 --> 00:41:38,560
this or here's the information. 
Or is Cleric just there reading 

742
00:41:38,560 --> 00:41:40,600
everything? 
Because I'm wondering how you 

743
00:41:40,600 --> 00:41:44,320
don't have a lot of noise. 
Well, that's a great question. 

744
00:41:45,920 --> 00:41:49,840
Noise is inevitable. 
You will learn completely 

745
00:41:49,840 --> 00:41:55,640
useless things. 
This will happen, but I think to

746
00:41:55,640 --> 00:41:57,760
us like it's more about 
curation, right? 

747
00:41:57,760 --> 00:41:59,920
Like how do you like, we call it
rot? 

748
00:41:59,920 --> 00:42:04,760
I think it's pretty standard 
term at this point that you're 

749
00:42:04,760 --> 00:42:07,280
just going to have this sort of 
memories that you've learned 

750
00:42:07,280 --> 00:42:12,680
which aren't very helpful. 
I mean, there's a couple of 

751
00:42:12,680 --> 00:42:16,000
ways. 
Like 1 is we give people an 

752
00:42:16,000 --> 00:42:18,320
explicit way to create them and 
forget them. 

753
00:42:18,800 --> 00:42:21,640
So if it's like we totally 
screwed it up, we learned 

754
00:42:21,640 --> 00:42:25,000
something we shouldn't have, or 
like we're somehow not learning 

755
00:42:25,000 --> 00:42:27,760
something we should, then you 
can just take over the wheel and

756
00:42:27,760 --> 00:42:30,120
be like, yo, you need to know 
this. 

757
00:42:30,120 --> 00:42:32,440
Like remember so. 
It's like, yeah, commit this to 

758
00:42:32,440 --> 00:42:35,120
memory, yeah. 
Or forget this exactly 

759
00:42:35,120 --> 00:42:36,280
eternally. 
Exactly. 

760
00:42:36,600 --> 00:42:39,360
Yeah, I don't do not mention 
this to me again. 

761
00:42:41,040 --> 00:42:45,320
But like that's usually for 
extremes, right, Because people 

762
00:42:45,320 --> 00:42:47,560
are lazy. 
Like you got a design for that, 

763
00:42:47,560 --> 00:42:50,640
Like you, you don't want to make
people say remember this all the

764
00:42:50,640 --> 00:42:53,240
time. 
So we do implicitly learn as 

765
00:42:53,240 --> 00:42:55,800
much as we can like from 
people's conversations. 

766
00:42:56,240 --> 00:42:59,720
We've also tried to make it 
really cheap for people to, you 

767
00:42:59,720 --> 00:43:02,880
know, say goodbye cleric and bad
boy like with ratings and things

768
00:43:02,880 --> 00:43:04,320
like that. 
Because then that also gives a 

769
00:43:04,320 --> 00:43:08,800
signal that all the decisions 
that we made so far, if it's a 5

770
00:43:08,800 --> 00:43:11,760
out of five, it's pretty good 
chance that we've did the right 

771
00:43:11,760 --> 00:43:13,640
thing, right? 
So then we can look through it. 

772
00:43:14,000 --> 00:43:16,600
Whereas if it's like a one out 
of five, even though we're like,

773
00:43:16,640 --> 00:43:19,400
oh, we did a really good job, 
then you know, OK, something 

774
00:43:19,400 --> 00:43:20,720
went wrong. 
So either that's something we 

775
00:43:20,720 --> 00:43:24,280
missed or like something we, you
know, over indexed on or 

776
00:43:24,280 --> 00:43:26,960
something like that. 
So that's that also helps a lot 

777
00:43:26,960 --> 00:43:31,320
from the learning. 
Back, continuing with the noise 

778
00:43:31,320 --> 00:43:36,760
theme, I imagine you can and 
never be satisfied on all the 

779
00:43:36,760 --> 00:43:42,360
data that you're collecting like
you, whether it comes to all 

780
00:43:42,360 --> 00:43:46,480
right, you've got your data dog,
you've got your code, you've got

781
00:43:46,480 --> 00:43:50,200
the Slack conversations, you've 
got the post mortem transcripts,

782
00:43:50,200 --> 00:43:53,920
you've got everything and 
anything that you can get your 

783
00:43:53,920 --> 00:43:57,200
hands on. 
Are there, is there a little bit

784
00:43:57,200 --> 00:44:00,880
of like a 8020 principle where 
you say, you know, what if we 

785
00:44:00,880 --> 00:44:05,280
just get the Slack conversation 
and the commit history, that 

786
00:44:05,280 --> 00:44:11,960
gets us pretty far. 
Yeah, I'd say Slack's amazing, 

787
00:44:11,960 --> 00:44:14,120
code's amazing, and logs are 
amazing. 

788
00:44:15,080 --> 00:44:21,240
These three get you very far. 
Slack mostly for just historical

789
00:44:21,240 --> 00:44:23,720
stuff. 
And this is again, like with the

790
00:44:23,720 --> 00:44:26,840
assumption that you haven't been
very disciplined and storing 

791
00:44:26,840 --> 00:44:28,920
everything. 
Like, you know, there are some 

792
00:44:28,920 --> 00:44:31,240
folks we work with who've that's
done a really good job. 

793
00:44:31,240 --> 00:44:33,280
So, you know, if you go to that 
confluence page, it's like 

794
00:44:33,280 --> 00:44:36,800
really well laid out, like all 
their lessons learned are in a 

795
00:44:37,080 --> 00:44:39,280
separate database. 
So then that's like obviously 

796
00:44:39,280 --> 00:44:42,600
much better. 
But generally like, yeah, like 

797
00:44:42,600 --> 00:44:46,600
this kind of text based 
knowledge is helpful. 

798
00:44:47,160 --> 00:44:53,080
The code is interesting because 
you're, you're looking for kind 

799
00:44:53,080 --> 00:44:56,160
of like a source of truth kind 
of knowledge, because no matter 

800
00:44:56,160 --> 00:45:01,560
what you see on Slack or 
documents or things like that, 

801
00:45:02,400 --> 00:45:06,720
you never really know how up to 
date it is, right? 

802
00:45:06,800 --> 00:45:11,600
Like, you know, I, I could say 
something today and a commit 

803
00:45:11,880 --> 00:45:15,000
tonight could completely 
invalidate that, right? 

804
00:45:15,000 --> 00:45:17,200
Like completely change 
everything because we've done a 

805
00:45:17,200 --> 00:45:21,880
big refactor. 
And so you still need to have 

806
00:45:21,880 --> 00:45:24,440
that source of truth. 
So it's like, do me to us like 

807
00:45:24,440 --> 00:45:27,080
code and Kubernetes, like 
infrastate is really helpful 

808
00:45:27,080 --> 00:45:29,000
because the agent can just be 
like, OK, hang on. 

809
00:45:29,000 --> 00:45:33,320
Like, like what is true today? 
And then like, can I work 

810
00:45:33,320 --> 00:45:35,640
backwards to, OK, what do I know
historically? 

811
00:45:35,640 --> 00:45:38,240
And then like, how do I you, you
can't go the other way around. 

812
00:45:38,240 --> 00:45:40,920
Like you're trying to piece 
together a truth from history. 

813
00:45:41,280 --> 00:45:46,200
It's hard, yeah. 
Now, the thing that William 

814
00:45:46,200 --> 00:45:51,440
always tells me when I talk to 
him, it's like it's pretty easy 

815
00:45:51,440 --> 00:45:57,040
to get something up and running 
in terms of an SRE agent that 

816
00:45:57,040 --> 00:46:01,600
you can demo. 
Yeah, like all 100% any any 

817
00:46:01,600 --> 00:46:04,800
agents basically. 
But I get it overnight and then,

818
00:46:04,800 --> 00:46:06,160
yeah, look, look pretty 
convincing. 

819
00:46:06,160 --> 00:46:09,040
And then he's like, but dude, 
when you get into real 

820
00:46:09,040 --> 00:46:16,200
production systems, it is so 
much harder to get an SRE agent 

821
00:46:16,240 --> 00:46:19,480
going. 
Can you explain like why is 

822
00:46:19,480 --> 00:46:21,160
that? 
What's the difference there? 

823
00:46:21,160 --> 00:46:24,720
That is the production gap that 
you see. 

824
00:46:26,280 --> 00:46:30,600
I'd say it's two things. 
One is just a human element. 

825
00:46:31,520 --> 00:46:34,280
The expectations are so much 
higher, right? 

826
00:46:34,320 --> 00:46:37,960
Because I mean, the way I like 
to think about it is if it's a 

827
00:46:37,960 --> 00:46:45,800
coding agent and I got 70% of 
the code right as a user, you 

828
00:46:45,800 --> 00:46:48,880
know, you just save me 70% of 
the keystrokes I would have to 

829
00:46:48,880 --> 00:46:50,800
make. 
I'm, I'm pretty happy, right? 

830
00:46:50,800 --> 00:46:54,920
Like I can, you know, look 
through it, even if it's not 

831
00:46:54,920 --> 00:46:57,760
right, it's, it's kind of cheap 
for me to know that it's not 

832
00:46:57,760 --> 00:46:59,880
correct. 
I can write some tests. 

833
00:46:59,880 --> 00:47:01,360
I can, I can get it over the 
line. 

834
00:47:01,360 --> 00:47:06,160
I'm not, I'm not pissed off with
you for being 70%, but we're on 

835
00:47:06,160 --> 00:47:09,160
the other extreme, right? 
Like, you know, if I'm 70% 

836
00:47:09,160 --> 00:47:13,920
correct and I LED you down this 
path, which is not exactly 

837
00:47:13,920 --> 00:47:16,960
right, and you've spent 20-30 
minutes in a pretty stressful 

838
00:47:16,960 --> 00:47:21,200
situation, sometimes you're 
going to hate me, right? 

839
00:47:21,200 --> 00:47:24,240
Like like forget about tech. 
It's just from a user point of 

840
00:47:24,240 --> 00:47:27,320
view like that's. 
Yeah, right. 

841
00:47:27,560 --> 00:47:29,680
Like why did you lie to me? 
Right. 

842
00:47:30,160 --> 00:47:33,040
And you know, and. 
That's actually, yeah. 

843
00:47:33,120 --> 00:47:35,200
I like to think about it. 
Yeah. 

844
00:47:35,200 --> 00:47:36,200
Yeah. 
You know, like the, like the 

845
00:47:36,200 --> 00:47:40,920
personality of people who are in
the, you know, the, the, the 

846
00:47:42,720 --> 00:47:44,920
like, the, you know, it's like 
the, the beast of the belly, 

847
00:47:44,920 --> 00:47:46,720
right? 
Like you're just the belly of 

848
00:47:46,720 --> 00:47:52,000
the beast belly. 
You, you, you get a certain kind

849
00:47:52,000 --> 00:47:54,560
of personality, right? 
Like you're not going to 

850
00:47:54,560 --> 00:47:56,880
tolerate, like you're very risk 
averse. 

851
00:47:56,880 --> 00:47:58,400
Yeah. 
Adrenaline's pumping. 

852
00:47:59,000 --> 00:48:01,200
Right. 
Because you know, like basically

853
00:48:01,200 --> 00:48:03,960
the kind of companies we work 
with are larger companies 

854
00:48:03,960 --> 00:48:06,480
because like if you're a 
startup, you don't really have a

855
00:48:06,480 --> 00:48:08,880
production problem, like you 
don't have that many customers. 

856
00:48:08,880 --> 00:48:11,840
But once you have a lot of 
customers, you're going to 

857
00:48:11,840 --> 00:48:14,920
allocate a certain set of people
to look after them. 

858
00:48:14,920 --> 00:48:17,440
And these people are like, 
obviously for very good reason, 

859
00:48:17,440 --> 00:48:21,280
like very, very careful. 
But on the then when you bring 

860
00:48:21,280 --> 00:48:24,760
into the technical side, that's 
also where the complexity is 

861
00:48:24,760 --> 00:48:31,440
because you know, for I'll just 
take code again because I think 

862
00:48:31,440 --> 00:48:33,760
most folks probably listen to 
this, are using coding agents. 

863
00:48:34,840 --> 00:48:40,920
Most of the truth that you need 
is in your code, right? 

864
00:48:40,920 --> 00:48:44,160
Like like for for a coding 
agent, you're actually going 

865
00:48:44,160 --> 00:48:46,240
deep. 
You give it this one thing, 

866
00:48:46,240 --> 00:48:49,040
you're coding repository, maybe 
it's couple and you're saying go

867
00:48:49,040 --> 00:48:51,920
build me this new feature. 
The complexity of the coding 

868
00:48:51,920 --> 00:48:54,720
agent is the kind of inputs of 
the tasks you give the coding 

869
00:48:54,720 --> 00:48:57,040
agent is potentially infinite. 
Like you can ask it to build 

870
00:48:57,120 --> 00:49:01,080
anything you want, but the base 
it's doing it on is finite, 

871
00:49:01,320 --> 00:49:04,360
right? 
It's just with SRE, it's the 

872
00:49:04,360 --> 00:49:09,600
opposite, right? 
The the answer could be anywhere

873
00:49:10,600 --> 00:49:12,640
so that there's no base like 
it's a, it's a, it's a 

874
00:49:12,640 --> 00:49:14,200
horizontal kind of breadth first
search. 

875
00:49:14,200 --> 00:49:16,960
But the kind of problems are 
kind of finite, like it's a 

876
00:49:16,960 --> 00:49:19,400
latency spike. 
It's a, you know, something's 

877
00:49:19,400 --> 00:49:21,720
down, the user's upset there's a
bug. 

878
00:49:22,000 --> 00:49:26,280
It's not like, so the complexity
really is like from a technical 

879
00:49:26,280 --> 00:49:29,160
standpoint that like the, it's a
needle in the air stack, like it

880
00:49:29,160 --> 00:49:31,920
could be anywhere. 
And so when you couple that with

881
00:49:31,920 --> 00:49:34,800
the user problem, hopefully that
gives you a sense of why this 

882
00:49:34,800 --> 00:49:36,680
is. 
Yeah, flip that right on its 

883
00:49:36,680 --> 00:49:39,120
head, dude. 
Oh, man. 

884
00:49:39,600 --> 00:49:45,840
One of the big questions that I 
am sure you think about a lot is

885
00:49:45,840 --> 00:49:49,320
like, are we just going to end 
up becoming a skill? 

886
00:49:51,880 --> 00:49:55,720
Is this whole thing going to get
so good that now we're a 

887
00:49:55,720 --> 00:49:58,720
markdown file that's part of a 
coding agent? 

888
00:49:59,520 --> 00:50:06,280
You don't like my default mode 
is I'm just a very optimistic 

889
00:50:06,280 --> 00:50:08,800
guy. 
So that's probably like a good 

890
00:50:08,800 --> 00:50:11,640
thing to just put it out there. 
Like that's why I think Willem 

891
00:50:11,640 --> 00:50:14,280
and I work really well because 
he's definitely a lot more like 

892
00:50:14,280 --> 00:50:17,240
on the like he'll see all the 
risks, like you know him well, 

893
00:50:17,240 --> 00:50:19,520
right? 
So I see all the optimism and 

894
00:50:19,520 --> 00:50:21,000
then we're trying to like meet 
in the middle. 

895
00:50:23,120 --> 00:50:25,680
I think that's part of why we 
should be having this this 

896
00:50:25,680 --> 00:50:27,760
discussion and like people like 
I should be talking about this 

897
00:50:27,760 --> 00:50:30,600
more because like the whole 
point is to avoid that future, 

898
00:50:30,640 --> 00:50:33,040
right? 
Like that's why I think, you 

899
00:50:33,040 --> 00:50:37,200
know, redefining how you see 
yourself as this Craftsman, as 

900
00:50:37,200 --> 00:50:39,760
this manager and like, where are
you actually getting value? 

901
00:50:39,840 --> 00:50:43,200
And just knowing where the value
is and just going for that. 

902
00:50:43,960 --> 00:50:46,080
That's what you got to do, 
right? 

903
00:50:46,080 --> 00:50:51,120
Because you know, you know, you 
just think about like there's 2 

904
00:50:51,400 --> 00:50:55,280
problems that obviously there's 
many, but I'm trying to reduce 

905
00:50:55,280 --> 00:50:57,600
it to 2 problems that the AI 
world is working on. 

906
00:50:57,960 --> 00:51:02,440
The first one is like, can you 
produce output that's, you know,

907
00:51:02,440 --> 00:51:05,840
like problem free, like no bugs.
It's you know, what I'm saying 

908
00:51:05,840 --> 00:51:09,080
is generally correct or rather 
like it, it works. 

909
00:51:09,520 --> 00:51:13,680
And the other one is like, is 
the work you're doing correct in

910
00:51:13,680 --> 00:51:15,720
that? 
Like is it directionally what, 

911
00:51:15,720 --> 00:51:17,720
what you want? 
Like is it a, should I build a 

912
00:51:17,720 --> 00:51:19,680
Mac OS app or should I build a 
web app? 

913
00:51:20,120 --> 00:51:22,160
Like that takes subjectivity and
taste. 

914
00:51:22,400 --> 00:51:25,600
Whereas the first one is more 
like, you know, if I asked you 

915
00:51:25,600 --> 00:51:27,200
to do a thing, did you do it 
right? 

916
00:51:28,880 --> 00:51:32,960
If your whole world is around 
the first problem, I mean, I've,

917
00:51:33,280 --> 00:51:34,640
there's, I don't have good news 
for you. 

918
00:51:34,640 --> 00:51:39,520
Like, like it is you have to 
assume that AI will just get so 

919
00:51:39,520 --> 00:51:43,280
good at solving that first 
problem that you need to think 

920
00:51:43,280 --> 00:51:45,680
about like, OK, what happens 
with the second problem, which 

921
00:51:45,680 --> 00:51:48,560
is like, you know, that's where 
the subjectivity and taste piece

922
00:51:48,560 --> 00:51:51,280
comes in and the real world 
constraints that you can't just 

923
00:51:51,280 --> 00:51:53,560
build everything overnight. 
People can't take it. 

924
00:51:54,480 --> 00:51:57,160
Then I think you want to be 
like, how do I be the purveyor 

925
00:51:57,160 --> 00:51:59,920
of taste and how do I be the one
who's really good at, like 

926
00:51:59,920 --> 00:52:03,240
making these decisions? 
Because I have this thing that 

927
00:52:03,240 --> 00:52:08,080
can actually produce working 
artifacts, but how do I make 

928
00:52:08,080 --> 00:52:09,800
sure it's better correct working
ones? 

929
00:52:10,480 --> 00:52:16,520
And continuing along those 
lines, do you ever feel like 

930
00:52:17,440 --> 00:52:20,520
Cleric could just be abstracted 
into a skill? 

931
00:52:21,800 --> 00:52:23,680
Could be. 
I mean I, I think. 

932
00:52:24,560 --> 00:52:27,800
Like. 
I think our job as founders is 

933
00:52:27,800 --> 00:52:32,160
like to constantly think about 
what is a durable problem and 

934
00:52:32,880 --> 00:52:36,040
what is a problem that you 
should not be spending an iota 

935
00:52:36,040 --> 00:52:39,840
of time on, right? 
Like, like when we started my, 

936
00:52:40,040 --> 00:52:43,640
my big rule to everybody was 
like, we are not fine tuning 

937
00:52:43,640 --> 00:52:47,400
models. 
Like this is like ludicrous. 

938
00:52:48,520 --> 00:52:51,040
People will figure this out. 
They're going to get way better 

939
00:52:51,040 --> 00:52:52,560
at like building way better 
models. 

940
00:52:53,360 --> 00:52:54,840
Just assume the models will get 
better. 

941
00:52:54,840 --> 00:52:57,600
I know you hate screaming at the
model with JS, like please give 

942
00:52:57,600 --> 00:52:59,160
me JS on, but this will be 
fixed. 

943
00:52:59,160 --> 00:53:01,320
Don't worry. 
Like spend time on the agent. 

944
00:53:01,840 --> 00:53:03,840
And then our whole thing like 
last year was like the agent 

945
00:53:03,840 --> 00:53:05,520
layers commoditized. 
Like it's a waste of time. 

946
00:53:05,520 --> 00:53:08,520
Like I think there's going to be
way better harnesses than we can

947
00:53:08,520 --> 00:53:10,200
ever build. 
There's going to be like better 

948
00:53:10,200 --> 00:53:12,480
agents and all that stuff. 
So then, you know, what we're 

949
00:53:12,480 --> 00:53:14,880
thinking about is like very much
on, you know, the verifiability 

950
00:53:14,880 --> 00:53:15,880
and learning and things like 
that. 

951
00:53:17,120 --> 00:53:19,160
I think the same time next year 
I'm going to be saying something

952
00:53:19,160 --> 00:53:20,760
different. 
Like that's the point, right? 

953
00:53:20,760 --> 00:53:23,760
Like you don't build something 
that everybody else is going to 

954
00:53:23,760 --> 00:53:26,080
do better than you. 
Like build a thing that's hard 

955
00:53:26,080 --> 00:53:27,720
in your domain that's going to 
last. 

956
00:53:27,720 --> 00:53:30,280
And like, I think the really 
tricky thing with AI is that 

957
00:53:30,280 --> 00:53:33,680
that thing is changing every 12 
months, but then you got to be 

958
00:53:33,680 --> 00:53:35,640
primed to keep making that jump 
every time. 

959
00:53:36,160 --> 00:53:40,360
You will talk to me about what 
2027 looks like it you kind of 

960
00:53:40,360 --> 00:53:42,640
mapped that out in this blog 
post that you wrote. 

961
00:53:42,840 --> 00:53:44,080
Right. 
I was trying to think about like

962
00:53:44,080 --> 00:53:50,520
what this world looks like. 
And to me it's almost like, you 

963
00:53:50,520 --> 00:53:54,920
know, you're, you're waking up 
and you've, you've had all these

964
00:53:54,920 --> 00:53:57,600
agents that have been working, 
you know, for you and then 

965
00:53:57,600 --> 00:53:59,640
they're sort of surfacing 
exceptions. 

966
00:53:59,960 --> 00:54:04,440
So maybe your coding agent 
shipped a change like to a 

967
00:54:04,440 --> 00:54:10,920
config and something ended up 
breaking at 2:00 AM and your SRE

968
00:54:10,960 --> 00:54:17,160
agent notices this reverses the 
change and watches what happened

969
00:54:17,200 --> 00:54:19,800
until the metrics seem to be 
fine. 

970
00:54:20,240 --> 00:54:22,760
And then you when you're waking 
up, you're just getting a report

971
00:54:22,760 --> 00:54:24,560
that you know, this is what 
happened. 

972
00:54:24,600 --> 00:54:28,200
Here's the decisions I took. 
Like, you know, I noticed that 

973
00:54:28,200 --> 00:54:32,640
the coding agent did X, so I 
decided that this was wrong, so 

974
00:54:32,640 --> 00:54:35,480
I reversed it. 
And so, you know, hope that's 

975
00:54:35,480 --> 00:54:37,160
OK, Don't worry like 
everything's fine. 

976
00:54:37,360 --> 00:54:42,320
Or it could be like it's about 
to make a mistake because it 

977
00:54:42,320 --> 00:54:44,640
doesn't know. 
So it's like surfacing kind of 

978
00:54:44,800 --> 00:54:47,680
like a question to you, like 
should I roll back? 

979
00:54:47,680 --> 00:54:49,600
Because I don't know, this is 
uncharted territory. 

980
00:54:50,120 --> 00:54:53,000
But that's almost like to me, 
the interface that we will be 

981
00:54:53,000 --> 00:54:56,800
operating with these agents. 
So to me, there's kind of two 

982
00:54:56,800 --> 00:55:00,600
interesting parts there. 
One is I call it like the 

983
00:55:00,600 --> 00:55:04,960
one-on-one problem, which is 
like as a human, like at what 

984
00:55:04,960 --> 00:55:08,200
level am I doing my one on ones 
with my agents? 

985
00:55:08,360 --> 00:55:10,920
And that's like the policies 
that we're setting, right? 

986
00:55:10,920 --> 00:55:14,240
So you know, like it's OK to 
roll back if X happens. 

987
00:55:14,240 --> 00:55:16,400
This is the way we build things,
things like that. 

988
00:55:17,200 --> 00:55:19,560
And then the other one, I call 
it like the stand up problem, 

989
00:55:19,560 --> 00:55:22,520
which is I think it's going to 
get up maybe a little bit more 

990
00:55:22,520 --> 00:55:26,080
interesting where once you have 
teams of agents, I'm not sure 

991
00:55:26,080 --> 00:55:28,760
we've done a great job yet as an
industry to have these agents 

992
00:55:28,760 --> 00:55:32,480
working together, right? 
Like today, you know, Cleric's 

993
00:55:32,480 --> 00:55:34,640
an SRE agent. 
And I've just talked about how 

994
00:55:34,640 --> 00:55:37,800
it's totally different to the 
coding agent, but it's so 

995
00:55:37,800 --> 00:55:39,760
obvious that we have to work 
really well together. 

996
00:55:39,760 --> 00:55:41,280
Yeah, and how do you share 
context? 

997
00:55:41,280 --> 00:55:42,920
How do you share context things 
like? 

998
00:55:42,920 --> 00:55:47,920
That how do you work off of the 
same planning, right And you're 

999
00:55:47,920 --> 00:55:52,560
able to cross one thing off or 
yeah, that's that is a really 

1000
00:55:52,560 --> 00:55:56,680
interesting one with with the 
proper isolation, right. 

1001
00:55:57,400 --> 00:56:00,960
So it's a little bit tricky in 
that regard because you want it 

1002
00:56:00,960 --> 00:56:03,520
to be isolated, but you want it 
to also share and share 

1003
00:56:03,520 --> 00:56:08,200
resources when needed. 
And so it gets, yeah, messy. 

1004
00:56:09,120 --> 00:56:12,360
I mean, I, I think, I think 
what's been really interesting 

1005
00:56:12,360 --> 00:56:15,480
about just thinking about agents
is that there's so much overlap 

1006
00:56:15,480 --> 00:56:18,520
with like how we've solved the 
same problem for humans. 

1007
00:56:19,400 --> 00:56:21,160
And like, you can borrow a lot 
of that stuff. 

1008
00:56:21,160 --> 00:56:24,440
Like, I don't think we need to 
reinvent the wheel in this case.

1009
00:56:24,440 --> 00:56:26,240
You know, you forget about 
agents. 

1010
00:56:26,240 --> 00:56:27,800
Like I've got a product 
engineer. 

1011
00:56:28,360 --> 00:56:32,040
A product engineer's main motive
is to increase the value of the 

1012
00:56:32,040 --> 00:56:34,720
product to their customer. 
They're not. 

1013
00:56:35,160 --> 00:56:38,080
I mean, hopefully they are, but 
they generally don't think about

1014
00:56:38,080 --> 00:56:40,800
security and reliability. 
They're just like, ship this as 

1015
00:56:40,800 --> 00:56:45,320
fast as I can get my promotion. 
And it's always like, whoa, what

1016
00:56:45,320 --> 00:56:47,960
are you doing there? 
Slow down like this. 

1017
00:56:47,960 --> 00:56:50,120
This could break something and 
that's going to be bad. 

1018
00:56:50,320 --> 00:56:52,120
Security will have the same 
thing, right? 

1019
00:56:52,120 --> 00:56:54,080
But then like, how do they work 
together? 

1020
00:56:54,080 --> 00:56:57,440
So, you know, there'll be 
something like if it does a big 

1021
00:56:57,440 --> 00:56:59,600
change, maybe the product 
engineer will be like, hey, I'm 

1022
00:56:59,600 --> 00:57:01,520
shipping APR. 
Can one of you guys have a look 

1023
00:57:01,520 --> 00:57:04,960
and let me know what you think? 
Pretty sure it's OK, but you 

1024
00:57:04,960 --> 00:57:07,280
know, let me know. 
There'll probably be some 

1025
00:57:07,280 --> 00:57:10,440
conventions that the SRE has 
written, like here's how you do 

1026
00:57:10,440 --> 00:57:11,960
incidents, here's how you 
investigate. 

1027
00:57:11,960 --> 00:57:14,800
Yeah, I was going to say that 
that feels very synchronous. 

1028
00:57:15,280 --> 00:57:19,760
And there's probably a world 
where it's happening in parallel

1029
00:57:19,760 --> 00:57:22,800
or it's they're happening 
together or yeah, like 

1030
00:57:23,160 --> 00:57:26,520
conventions might be a very 
rudimentary way of looking at 

1031
00:57:26,520 --> 00:57:30,280
that. 
Or they're just one is writing 

1032
00:57:30,280 --> 00:57:32,920
the code and the next thing you 
know, there's not even like APR 

1033
00:57:32,920 --> 00:57:35,640
that needs to happen because one
writes the code, the other one 

1034
00:57:35,640 --> 00:57:38,960
comes and refractors it as it's 
happening, right? 

1035
00:57:40,280 --> 00:57:43,000
But I think like, at least on 
the human side, it's more the 

1036
00:57:43,000 --> 00:57:46,480
recognition that you have these 
two entities who are experts in 

1037
00:57:46,480 --> 00:57:50,920
their own domain. 
And we've found very slow ways 

1038
00:57:50,920 --> 00:57:53,600
for them to talk to each other. 
But like we, we agree that these

1039
00:57:53,600 --> 00:57:55,600
are two separate entities 
because they're very separate 

1040
00:57:55,600 --> 00:57:57,880
objectives. 
So they they do belong in the 

1041
00:57:57,880 --> 00:58:01,960
org, but you know, they've got 
to talk. 

1042
00:58:02,440 --> 00:58:06,680
So I don't, I just think that, 
you know, by 2027 we'd 

1043
00:58:06,680 --> 00:58:08,320
definitely, whether I figure 
this out, like I know, like 

1044
00:58:08,320 --> 00:58:10,360
we're thinking about this 
deeply, like how do we integrate

1045
00:58:10,360 --> 00:58:13,560
even better with, you know, the 
cloud codes and the codexors of 

1046
00:58:13,560 --> 00:58:16,440
the world? 
And how do we actually bring 

1047
00:58:16,440 --> 00:58:20,520
back all the knowledge that 
we've gained from production so 

1048
00:58:20,520 --> 00:58:23,800
that, you know, you're giving it
to the coding agent. 

1049
00:58:23,800 --> 00:58:25,400
So. 
The coding agent doesn't make 

1050
00:58:25,400 --> 00:58:28,600
the same mistake exactly. 
Yeah, that to me is value, 

1051
00:58:28,600 --> 00:58:31,400
right. 
Like that's, you know, like one 

1052
00:58:31,400 --> 00:58:36,120
of the things which is like my 
big bet beeve is, you know, the 

1053
00:58:36,120 --> 00:58:40,200
whole point of SRE is you're 
trying to protect the 

1054
00:58:40,200 --> 00:58:44,160
reliability for your customers. 
It's not to reduce incidence. 

1055
00:58:44,160 --> 00:58:46,520
Like that's a byproduct, right? 
Like the whole point is like 

1056
00:58:46,520 --> 00:58:49,600
that's your main goal. 
So if you can prevent the 

1057
00:58:49,600 --> 00:58:51,720
incident, that's what you should
be doing, right? 

1058
00:58:51,720 --> 00:58:56,280
So seeing like any model where 
it's like, oh, I improve MPTR 

1059
00:58:56,280 --> 00:58:59,200
and I've like I'm on jumping on 
every incident my F. 

1060
00:58:59,280 --> 00:59:03,080
One score, it's just like the 
back in the day with the machine

1061
00:59:03,080 --> 00:59:06,600
learning models and being like, 
wow, my F1 score's amazing. 

1062
00:59:06,600 --> 00:59:09,600
And then you're like, actually 
nobody uses that model, so. 

1063
00:59:09,680 --> 00:59:11,360
Where you missed the prime. 
Yeah, exactly. 

1064
00:59:11,600 --> 00:59:16,840
Exactly. 
Oh man, I did want to ask you 

1065
00:59:16,840 --> 00:59:21,560
about how you're bringing the 
learnings back from production 

1066
00:59:21,800 --> 00:59:25,440
into the coding agents. 
So I, I mean, I, I think it's 

1067
00:59:25,440 --> 00:59:29,920
rudimentary right now. 
So today you can have the coding

1068
00:59:29,920 --> 00:59:34,760
agent speak to Cleric through 
MCP and, you know, you kind of 

1069
00:59:34,760 --> 00:59:39,000
like get Cleric to weigh in on 
what it's on, what he thinks. 

1070
00:59:40,480 --> 00:59:44,200
We also have, you know, like 
these conventions that you can 

1071
00:59:44,200 --> 00:59:46,680
have in the repository, which we
know that they're coding agents 

1072
00:59:46,680 --> 00:59:52,000
also use things like that. 
But I don't think that that's 

1073
00:59:52,000 --> 00:59:54,880
where it's going to end. 
I think we, you know, like 

1074
00:59:54,920 --> 00:59:58,520
there's already kind of 
attempts. 

1075
00:59:58,520 --> 01:00:01,720
I know that Google's working on 
A to a I'm not sure how much 

1076
01:00:01,880 --> 01:00:04,760
like the adoption has been, but 
like you, we're trying to create

1077
01:00:04,760 --> 01:00:07,360
these open stands for these 
agents to talk to each other, 

1078
01:00:07,520 --> 01:00:09,960
right? 
We haven't adopted any of these 

1079
01:00:09,960 --> 01:00:12,760
yet, just to be upfront, but I 
think they will become more and 

1080
01:00:12,760 --> 01:00:15,920
more of a thing because to your 
point, you don't want it to be 

1081
01:00:15,920 --> 01:00:18,360
synchronous, you want it to be 
like info. 

1082
01:00:18,360 --> 01:00:20,680
Yeah. 
And how do you just keep it so 

1083
01:00:20,680 --> 01:00:27,160
that it's not blowing up the 
context when you do have 

1084
01:00:27,560 --> 01:00:31,280
whatever it is that you want to 
give it as much information 

1085
01:00:31,280 --> 01:00:34,720
about what you've learned from 
production, but you don't 

1086
01:00:34,720 --> 01:00:37,600
necessarily want to give it 
everything because then you're 

1087
01:00:37,600 --> 01:00:39,680
just blown up the context every 
time. 

1088
01:00:39,680 --> 01:00:44,440
And so is it where you need to 
even add it to the context or 

1089
01:00:44,440 --> 01:00:47,120
you just have that sub agent 
that's coming in behind every 

1090
01:00:47,120 --> 01:00:52,680
coding agent that is your sub 
agent, SRE agent that says, Oh, 

1091
01:00:52,680 --> 01:00:56,320
yeah, I know about these things.
And I know that this happened 

1092
01:00:56,320 --> 01:01:00,720
recently and it has all of the 
necessary information. 

1093
01:01:01,320 --> 01:01:06,200
And like, that also doesn't feel
like the right way to do it, you

1094
01:01:06,200 --> 01:01:11,000
know, so have some clean up 
agent that is constantly going 

1095
01:01:11,000 --> 01:01:16,680
behind the coding agent. 
And it's just like, but how do 

1096
01:01:16,680 --> 01:01:21,520
you make sure that you do get 
that 360 degree? 

1097
01:01:21,520 --> 01:01:25,720
You have the security, you have 
the, the stability, you have all

1098
01:01:25,720 --> 01:01:27,960
of these things that you're 
checking for. 

1099
01:01:29,360 --> 01:01:37,120
I, I've heard when we had Sid do
the coding agent conference, he 

1100
01:01:37,120 --> 01:01:40,760
said that one thing that they'll
have is an adversarial agent 

1101
01:01:41,400 --> 01:01:45,520
that will try and just, you 
know, find problems with 

1102
01:01:45,520 --> 01:01:47,880
whatever it is. 
And so that's one way that 

1103
01:01:47,880 --> 01:01:54,400
they'll do it, but really 
relaying the information from 

1104
01:01:54,400 --> 01:01:58,160
production to recognize that, 
hey, this happened. 

1105
01:01:58,680 --> 01:02:00,720
It's the second time that this 
has happened. 

1106
01:02:00,720 --> 01:02:03,640
Let's not have a third time, 
right, guys? 

1107
01:02:03,640 --> 01:02:10,840
Come on, make sure. 
I just, I want to see what the 

1108
01:02:10,840 --> 01:02:14,200
dynamics are of the agents that 
you wake up to. 

1109
01:02:14,200 --> 01:02:17,640
And the SRE agent is just Debbie
Downer all the time. 

1110
01:02:17,640 --> 01:02:24,840
Like you know this PM the agent 
is constantly shipping crap and 

1111
01:02:24,840 --> 01:02:27,520
I have to clean up after. 
What is this? 

1112
01:02:29,000 --> 01:02:31,920
Wait, you don't like that's 
happening now And it's already 

1113
01:02:31,920 --> 01:02:36,360
just, it's, it's hilarious. 
Like we, we have a sub agent 

1114
01:02:36,760 --> 01:02:41,920
that we call the investigation 
auditor and the investigation 

1115
01:02:41,920 --> 01:02:44,040
auditor's job is to be Debbie 
Downer. 

1116
01:02:44,200 --> 01:02:46,000
It's like you have not done a 
good enough job. 

1117
01:02:46,120 --> 01:02:48,080
Keep going. 
And he is like, why he's screwed

1118
01:02:48,080 --> 01:02:51,000
up and things like that. 
And then, you know, sometimes 

1119
01:02:51,000 --> 01:02:54,240
the main agent will complain 
saying like this is like too 

1120
01:02:54,240 --> 01:02:57,880
many Times Now I, I think I've 
done the best I've heard. 

1121
01:02:58,200 --> 01:03:00,080
So I'm just going to return back
to the user. 

1122
01:03:01,000 --> 01:03:03,440
Yeah, it's, it's, it's really 
funny just reading through these

1123
01:03:03,440 --> 01:03:06,920
traces. 
But it's not even mentioned that

1124
01:03:06,920 --> 01:03:09,200
because I think, you know, like 
stuff I talked about at the 

1125
01:03:09,200 --> 01:03:11,400
beginning, like, you know, the 
Ant colony and things like that.

1126
01:03:11,400 --> 01:03:14,600
Like, I think we've got to the 
point where we're now doing a 

1127
01:03:14,600 --> 01:03:17,280
lot of experimentation on what 
are the right abstractions, 

1128
01:03:17,280 --> 01:03:19,840
right? 
Because I think, you know, for a

1129
01:03:19,840 --> 01:03:23,320
long time it's been sort of the 
main agent, only then it was 

1130
01:03:23,320 --> 01:03:26,200
main agent, sub agent. 
Like I think and topics are 

1131
01:03:26,200 --> 01:03:28,560
releasing. 
I think it's still in beta where

1132
01:03:29,040 --> 01:03:32,840
you actually have like Asian 
teams, which are some totally 

1133
01:03:32,840 --> 01:03:35,480
different agents and they don't 
share any context and they're 

1134
01:03:35,480 --> 01:03:38,280
just like talking to each other 
through, you know, some form of 

1135
01:03:38,280 --> 01:03:40,720
protocol. 
So I think we'll just have more 

1136
01:03:40,720 --> 01:03:44,920
and more of these things. 
I think, I think to me like the 

1137
01:03:44,920 --> 01:03:49,760
coding agent, SRE agent, the 
team of agents paradigm makes 

1138
01:03:49,760 --> 01:03:53,240
much more sense because, you 
know, coding agent doesn't need 

1139
01:03:53,240 --> 01:03:55,160
to know the internals of the SRE
agent. 

1140
01:03:55,480 --> 01:03:57,720
The coding agent just need to 
know that, hey, that's his 

1141
01:03:57,720 --> 01:04:00,200
expert that I need to talk to 
you about. 

1142
01:04:00,920 --> 01:04:03,280
Yeah. 
And you know, but you know, I'm 

1143
01:04:03,280 --> 01:04:07,000
not, I'm not going to totally 
listen to this agent, but I just

1144
01:04:07,000 --> 01:04:10,920
want to get some, you know, 
like, isn't that like a kind of 

1145
01:04:10,920 --> 01:04:13,320
a consulting thing where it's 
like the racing matrix, like 

1146
01:04:13,320 --> 01:04:17,080
responsible, accountable, 
consulted, informed. 

1147
01:04:17,520 --> 01:04:19,280
I think you'll have that for 
agents, right? 

1148
01:04:19,280 --> 01:04:23,480
Like, you know, you must listen 
to the SRV agent or no, you may 

1149
01:04:23,480 --> 01:04:25,800
consult the SRV agent and things
like that. 

1150
01:04:25,800 --> 01:04:28,200
So I think these kind of like 
governance structures that we 

1151
01:04:28,200 --> 01:04:30,320
come up with are going to be 
very, very valuable. 

1152
01:04:30,680 --> 01:04:38,160
Yeah, a lot of the software that
we use and love is going local 

1153
01:04:38,160 --> 01:04:41,640
1st. 
So as opposed to me building 

1154
01:04:41,640 --> 01:04:45,600
something and exposing it to the
world, I build something that I 

1155
01:04:45,600 --> 01:04:48,600
exposed to myself. 
And so because of that, I don't 

1156
01:04:48,600 --> 01:04:52,320
necessarily need to have all the
security and all of the sure, it

1157
01:04:52,320 --> 01:04:54,480
doesn't need to be battle 
hardened for production if it's 

1158
01:04:54,480 --> 01:04:58,160
just me as the main consumer 
because it's very bespoke 

1159
01:04:58,160 --> 01:05:03,880
software. 
And now, even more so, I'm 

1160
01:05:03,880 --> 01:05:08,080
looking at it like, wow, a lot 
of things that I used to build, 

1161
01:05:08,480 --> 01:05:10,760
I don't even need to build 
anymore because I just create 

1162
01:05:10,760 --> 01:05:12,520
them as a skill. 
Sure. 

1163
01:05:13,040 --> 01:05:18,160
And so it doesn't require a 
full-fledged app, it just 

1164
01:05:18,160 --> 01:05:22,160
requires a very clean skill, 
right? 

1165
01:05:22,200 --> 01:05:27,560
And so the, the act of moving 
from production grade software 

1166
01:05:27,560 --> 01:05:31,920
to skill, which is very, very 
thin. 

1167
01:05:31,920 --> 01:05:37,560
And it's like the lightest form 
of a application possible, 

1168
01:05:37,560 --> 01:05:39,160
right? 
It's just markdown, right? 

1169
01:05:39,560 --> 01:05:41,840
So that's, that's kind of how I 
look at it. 

1170
01:05:42,880 --> 01:05:49,240
Not to say that the software is 
not, but like, I still feel like

1171
01:05:49,240 --> 01:05:52,960
there's a lot of very valuable 
software out there. 

1172
01:05:53,680 --> 01:05:56,960
And so I'm not fully bought into
the software's cooked thing. 

1173
01:05:57,800 --> 01:06:00,280
There's just a class of software
that you just never going to 

1174
01:06:00,280 --> 01:06:06,400
write code in my view like. 
Payments processing, Stripe, 

1175
01:06:06,600 --> 01:06:08,520
you're not gonna. 
That that's a good one. 

1176
01:06:08,600 --> 01:06:11,840
But like, you know, some, some 
like like Big Query, for 

1177
01:06:11,840 --> 01:06:16,200
example, right? 
Like we dump a lot of our data 

1178
01:06:16,200 --> 01:06:19,320
in there just for analysis, 
things like that. 

1179
01:06:19,320 --> 01:06:23,200
I'm not gonna wipe code that 
like, I mean, it's just doesn't 

1180
01:06:23,200 --> 01:06:25,640
make any sense. 
Like the this is a product that 

1181
01:06:25,640 --> 01:06:27,840
probably took years and years 
and years to get right. 

1182
01:06:29,520 --> 01:06:32,520
And there's almost like a level 
of accuracy that you expect. 

1183
01:06:32,520 --> 01:06:36,280
Like when I write a query, I get
it exactly like what I want back

1184
01:06:36,280 --> 01:06:38,160
and it's performant and all that
stuff. 

1185
01:06:39,080 --> 01:06:41,840
So, you know, physics still 
applies, right? 

1186
01:06:41,840 --> 01:06:45,680
Like there's cost of having an 
engineer thinking about writing 

1187
01:06:45,680 --> 01:06:49,120
the software, making sure you're
maintaining it because it's, 

1188
01:06:49,120 --> 01:06:51,600
it's always been true with 
software that the easy part is 

1189
01:06:51,600 --> 01:06:53,240
building it. 
The hard part is maintaining it.

1190
01:06:53,240 --> 01:06:56,080
That's the reason we exist. 
You build a whole bunch of shit,

1191
01:06:56,080 --> 01:06:57,880
it's in production now, you got 
to maintain it. 

1192
01:06:57,880 --> 01:07:00,040
And now most of the engineers 
are spending time maintaining it

1193
01:07:00,040 --> 01:07:01,680
and not actually getting to 
create. 

1194
01:07:01,920 --> 01:07:05,360
So our whole value property is, 
you know, you can go create now 

1195
01:07:05,360 --> 01:07:08,080
because we can take some of that
work, but the cost is real, 

1196
01:07:08,280 --> 01:07:10,800
right? 
But yeah, you're right. 

1197
01:07:10,800 --> 01:07:14,880
Like there's going to be, you 
know, but some software where, I

1198
01:07:14,880 --> 01:07:17,320
don't know, there's like 20 
features, but you only need one.

1199
01:07:18,000 --> 01:07:20,440
Maybe that's just the skill. 
Maybe that's like a little app 

1200
01:07:20,440 --> 01:07:23,520
that you build. 
I think where things get more 

1201
01:07:23,520 --> 01:07:26,320
interesting to me is like in 
that, in that like kind of 

1202
01:07:26,320 --> 01:07:32,120
middle ground where I feel, and 
I'm pushing for this internally 

1203
01:07:32,120 --> 01:07:35,760
a lot where we want to identify 
everything we do. 

1204
01:07:37,040 --> 01:07:42,720
So I mean, I, I remember like 
like a year ago, William and I 

1205
01:07:42,720 --> 01:07:46,920
were just really frustrated by 
how long it took to get a 

1206
01:07:46,920 --> 01:07:50,480
website update done. 
It just, it was like, you have 

1207
01:07:50,480 --> 01:07:53,040
so many ideas, but I was like, 
you need a, there's a meeting, 

1208
01:07:53,040 --> 01:07:55,480
you know, talk to this person, 
that person, blah, blah, blah, 

1209
01:07:55,480 --> 01:07:58,000
blah, blah. 
And I think like a few months 

1210
01:07:58,000 --> 01:08:01,240
ago, like, all right, that's it.
We're moving to Astro. 

1211
01:08:01,600 --> 01:08:04,560
Everything's going to be code. 
A whole marketing site is 

1212
01:08:04,560 --> 01:08:07,520
basically an Astro site now. 
It's no longer on any kind of no

1213
01:08:07,520 --> 01:08:10,720
code platform. 
And man, it's amazing, like. 

1214
01:08:10,880 --> 01:08:13,000
You're just so much faster. 
So much faster. 

1215
01:08:13,000 --> 01:08:16,479
That's wild. 
It's and so so that's going to 

1216
01:08:16,479 --> 01:08:21,120
happen, right. 
Like it's like if your platform 

1217
01:08:21,399 --> 01:08:26,200
is not agent native in that 
agents like talking to you and 

1218
01:08:26,200 --> 01:08:30,000
working with you, then that's 
that's trouble. 

1219
01:08:30,840 --> 01:08:35,800
Yeah, right. 
The part that is worth 

1220
01:08:35,800 --> 01:08:38,000
considering is again, what's 
durable? 

1221
01:08:38,319 --> 01:08:40,680
What are the things that you're 
not going to vibe code? 

1222
01:08:41,120 --> 01:08:44,160
Sure, versus what are the things
that you could probably get away

1223
01:08:44,160 --> 01:08:48,640
with vibe coding? 
Or what are the things that you 

1224
01:08:48,960 --> 01:08:53,640
definitely need to be a gentic 
first or you want to be able to 

1225
01:08:53,640 --> 01:08:56,840
use with coding agents, You want
to be able to have that seamless

1226
01:08:56,840 --> 01:09:00,920
experience with coding agents. 
And so like looking at those 

1227
01:09:00,920 --> 01:09:03,439
that Spectrum is another good 
one. 

1228
01:09:03,439 --> 01:09:09,000
And I've, I feel like the system
of records thing that's probably

1229
01:09:09,760 --> 01:09:13,120
not going anywhere because 
that's pretty hard to do 

1230
01:09:13,120 --> 01:09:17,520
Databases, like you're saying, 
Big query is another hard one. 

1231
01:09:17,520 --> 01:09:20,359
Like what you really are going 
to go vibe code to database. 

1232
01:09:20,840 --> 01:09:24,120
There's very performing 
databases and they've been 

1233
01:09:24,200 --> 01:09:28,160
performing. 
Tested through like years and 

1234
01:09:28,160 --> 01:09:30,600
years and years, yeah, I think 
it's almost like a dimension 

1235
01:09:30,600 --> 01:09:33,120
thing, right? 
Which is like, how hard is it to

1236
01:09:33,120 --> 01:09:37,160
actually build? 
And what is the friction it 

1237
01:09:37,160 --> 01:09:39,960
creates for you to get an agent 
to work with it? 

1238
01:09:40,760 --> 01:09:44,880
And then if that friction is 
high enough and the amount of 

1239
01:09:44,880 --> 01:09:46,040
time you need to build it. 
Is. 

1240
01:09:46,040 --> 01:09:47,640
Oh, that's a nice little 
diagram, yeah. 

1241
01:09:48,000 --> 01:09:49,439
Now that's going to be white 
coated. 

1242
01:09:49,640 --> 01:09:54,200
I'm sorry, but it's it's gone. 
You're feeling that frustration 

1243
01:09:54,200 --> 01:09:58,080
of like, I really want my agent 
to be able to plug into this. 

1244
01:09:58,200 --> 01:10:01,120
Yeah. 
And could I build it? 

1245
01:10:01,440 --> 01:10:03,560
Let's try. 
All right, Because I want to be 

1246
01:10:03,560 --> 01:10:06,160
able to have something that my 
agent can plug into easily. 

1247
01:10:06,320 --> 01:10:09,640
If my agent builds it itself 
then it's definitely going to be

1248
01:10:09,640 --> 01:10:13,960
able to plug into it. 100% like,
I mean, I at this point, I mean 

1249
01:10:13,960 --> 01:10:17,640
I've, I've, you know, I have an 
agent that I've connected to 

1250
01:10:17,640 --> 01:10:22,360
like all my stuff like my Gmail,
my, you know, hotspot and 

1251
01:10:22,360 --> 01:10:25,400
everything. 
And I'm constantly looking for 

1252
01:10:25,400 --> 01:10:29,360
like OK how do I make this thing
faster and better? 

1253
01:10:29,400 --> 01:10:32,120
You know what the biggest one 
for me was? 

1254
01:10:32,200 --> 01:10:35,240
My buddy JQ said, hey, we got 
rid of linear. 

1255
01:10:35,600 --> 01:10:37,600
We just went to all markdown 
files. 

1256
01:10:37,840 --> 01:10:41,400
Our whole thing is all markdown 
files and agents are able to do 

1257
01:10:41,680 --> 01:10:45,240
whatever with the markdown and 
they update it and they keep 

1258
01:10:45,240 --> 01:10:48,040
state on how the files are 
going. 

1259
01:10:48,600 --> 01:10:52,120
And so he's like, we realize 
like, what are we paying all his

1260
01:10:52,120 --> 01:10:55,200
money to linear for? 
We don't even need it because I 

1261
01:10:55,200 --> 01:10:58,760
don't he he was saying, I 
realized I wasn't going into 

1262
01:10:58,760 --> 01:11:03,240
linear and looking at it and all
I was doing is asking the agent,

1263
01:11:03,240 --> 01:11:05,040
Hey, what's the status on this 
linear issue? 

1264
01:11:05,640 --> 01:11:09,160
And so there was no need to 
actually have linear. 

1265
01:11:09,440 --> 01:11:13,240
But then the funniest part and 
it's full circle moment, he 

1266
01:11:13,240 --> 01:11:17,520
said, yeah, so now we realize 
there are people that like to 

1267
01:11:17,520 --> 01:11:21,400
see the status. 
So we're now vibe coding a front

1268
01:11:21,400 --> 01:11:24,000
end. 
So I'm like, wait a minute. 

1269
01:11:24,120 --> 01:11:27,160
So I mean the irony would be if 
you, I guess that's what you're 

1270
01:11:27,160 --> 01:11:29,680
implying with going full circle,
you're kind of building linear. 

1271
01:11:30,200 --> 01:11:34,720
Yeah, in a way that is much more
lightweight and he doesn't 

1272
01:11:34,720 --> 01:11:39,360
necessarily need like linear. 
I thought it was durable, but it

1273
01:11:39,800 --> 01:11:41,840
at the end of the day it's not 
really durable, right. 

1274
01:11:41,840 --> 01:11:46,440
Like all of your, your status 
can be marked down and so if 

1275
01:11:46,440 --> 01:11:50,320
everything is marked down and 
then it can grab context from 

1276
01:11:50,400 --> 01:11:53,840
other markdown files. 
So if you're you really go all 

1277
01:11:53,840 --> 01:12:01,680
in on a file system, then why do
you need linear to be the middle

1278
01:12:01,680 --> 01:12:04,360
man? 
I would disagree that, but like 

1279
01:12:04,360 --> 01:12:06,080
I I think for everybody it's 
different. 

1280
01:12:06,080 --> 01:12:11,320
Like to me, in that same 
dimensions, linear is very low 

1281
01:12:11,320 --> 01:12:16,160
friction for an agent to use. 
I think the we use it 

1282
01:12:16,160 --> 01:12:19,440
internally. 
So although I do wish to release

1283
01:12:19,440 --> 01:12:23,680
an official CLI, I'm not a big 
fan of the MCP. 

1284
01:12:23,680 --> 01:12:28,560
But generally, there's so much 
training data on linear that 

1285
01:12:29,240 --> 01:12:30,720
agents are pretty good at using 
them. 

1286
01:12:31,600 --> 01:12:34,080
And yeah, the, the effort to 
wipe coat it is probably not 

1287
01:12:34,080 --> 01:12:35,560
that high. 
But since the friction is so 

1288
01:12:35,560 --> 01:12:39,440
low, you're just like, it's not 
really worth my time to be 

1289
01:12:39,480 --> 01:12:42,760
thinking about this thing. 
I'd rather go focus on something

1290
01:12:42,760 --> 01:12:47,000
else, right? 
So like to me, it's like the 

1291
01:12:47,000 --> 01:12:51,920
friction has to be really high 
that it's like actively impeding

1292
01:12:51,920 --> 01:12:55,960
me for me to think it's worth 
going ahead because and to the 

1293
01:12:55,960 --> 01:12:59,080
point now, am I building a 
linear alternative or am I 

1294
01:12:59,080 --> 01:13:01,200
building my business? 
That's what I'm thinking. 

1295
01:13:01,560 --> 01:13:08,200
I wonder if it is for JQ. 
The friction was just in him 

1296
01:13:08,200 --> 01:13:10,560
seeing the bill. 
Could be, yeah, it could be 

1297
01:13:10,560 --> 01:13:12,880
that. 
Was what he was like, man, what 

1298
01:13:12,880 --> 01:13:15,000
is this like it it just eats at 
him. 

1299
01:13:15,000 --> 01:13:18,680
And so maybe it was just like a 
a little bit of a side project 

1300
01:13:18,680 --> 01:13:22,120
that became a a bigger project 
and he realized wow, all right, 

1301
01:13:22,320 --> 01:13:25,400
let's yeah, let's just see. 
And if you look at the bill, I 

1302
01:13:25,440 --> 01:13:27,960
don't think it's that expensive,
right. 

1303
01:13:28,440 --> 01:13:31,880
So he really probably was like, 
let's just try. 

1304
01:13:31,880 --> 01:13:37,040
But he also is trying to get 
agents doing everything. 

1305
01:13:37,040 --> 01:13:41,840
Like he has five different names
of agents in his slack that 

1306
01:13:41,840 --> 01:13:45,800
he'll call on to do things. 
You know, because they're the 

1307
01:13:45,800 --> 01:13:49,840
different agents that oh, this 
is the whatever linear knock off

1308
01:13:49,840 --> 01:13:58,320
agent, right, right. 
Linney Linney Yeah, like, but 

1309
01:13:58,360 --> 01:14:01,240
but actually it's ironic that 
we're talking about linear 

1310
01:14:01,240 --> 01:14:04,320
because I think this CEO of 
linear, I don't I can't remember

1311
01:14:04,320 --> 01:14:07,520
his name, Carrie. 
I think he's probably got one of

1312
01:14:07,640 --> 01:14:10,760
like this court that I, I really
liked. 

1313
01:14:11,320 --> 01:14:17,240
He talked about how like a whole
point of building a company is 

1314
01:14:17,240 --> 01:14:21,280
like you got to be focused on 
your main quest side quest will 

1315
01:14:21,280 --> 01:14:23,960
keep coming, but you got to 
focus on your main quest. 

1316
01:14:23,960 --> 01:14:26,520
Like, that is like your number 
one thing as a as a founder. 

1317
01:14:27,080 --> 01:14:30,320
And the thing with the agents is
that the side quests have become

1318
01:14:30,320 --> 01:14:32,040
really tempting. 
Yeah. 

1319
01:14:32,480 --> 01:14:36,280
'Cause you can, you know, you 
look at like what you're, you're

1320
01:14:36,280 --> 01:14:38,560
like, I'm just like reflecting 
as you know. 

1321
01:14:38,560 --> 01:14:40,960
Yeah, what JQ did, he was like 
again. 

1322
01:14:41,080 --> 01:14:44,080
I don't know, like it could be 
like a so I can't I can't speak 

1323
01:14:44,080 --> 01:14:45,480
to like whether there was a side
quest or not. 

1324
01:14:45,480 --> 01:14:48,760
Like it seemed yeah, like I 
don't want to roast him on like 

1325
01:14:48,760 --> 01:14:51,800
something I don't know. 
But like one of the things which

1326
01:14:51,800 --> 01:14:56,760
I, I've I've seen is that being 
an engineer founder is actually 

1327
01:14:56,760 --> 01:14:59,800
one of the hardest things 
because like coding feels so 

1328
01:14:59,800 --> 01:15:01,960
good. 
You feel like you're being 

1329
01:15:01,960 --> 01:15:03,480
productive, you're getting shit 
done. 

1330
01:15:04,080 --> 01:15:07,360
And when you see stuff like 
this, you can be like, Oh, I can

1331
01:15:07,360 --> 01:15:09,160
help my team. 
I'll just, you know, build this 

1332
01:15:09,160 --> 01:15:11,560
thing. 
But then like to me, it's like, 

1333
01:15:11,560 --> 01:15:15,160
well, that's not what's going to
make Derrick successful. 

1334
01:15:15,280 --> 01:15:19,880
I I don't think, you know, like 
my team needs me to do like a so

1335
01:15:19,880 --> 01:15:24,400
the only thing like like we've 
really like wide coated 

1336
01:15:24,400 --> 01:15:31,280
seriously outside like cleric is
like this, like this one off 

1337
01:15:31,280 --> 01:15:35,440
kind of MCPS or CL is that makes
it easier for us to maybe like 

1338
01:15:35,440 --> 01:15:37,320
respond back to a customer or 
something like that. 

1339
01:15:37,880 --> 01:15:40,720
But all the tools that we're 
building is really either to 

1340
01:15:40,720 --> 01:15:45,680
improve the CICD or something 
like, but if it's does a thing, 

1341
01:15:45,680 --> 01:15:48,920
it's working and it's not crazy 
expensive science living. 

1342
01:15:49,960 --> 01:15:54,920
You know, it's the exact 
opposite of what my talk was in 

1343
01:15:54,920 --> 01:15:58,880
the coding agent conference, cuz
I, my whole talk was called 

1344
01:15:58,880 --> 01:16:00,880
enjoy the side quest. 
Right, right, right. 

1345
01:16:00,880 --> 01:16:06,000
And so it was saying how I gave 
the example of basically me 

1346
01:16:06,040 --> 01:16:12,560
being like, oh, so I wanted to 
build an app and I wanted to 

1347
01:16:12,760 --> 01:16:14,680
schedule it. 
So then I Start learning about 

1348
01:16:14,680 --> 01:16:18,800
like GitHub workflows and boom. 
So then that's cool. 

1349
01:16:18,800 --> 01:16:27,000
But then I wanted to make sure 
that it was using a local model 

1350
01:16:27,200 --> 01:16:31,760
because it started to get too 
expensive when I was sure doing 

1351
01:16:31,760 --> 01:16:34,920
this. 
So then I grabbed and I, I 

1352
01:16:34,920 --> 01:16:37,280
started learning about like 
quantizing models and run it on 

1353
01:16:37,280 --> 01:16:39,240
my own machine. 
Then I was like, ah, well, 

1354
01:16:39,520 --> 01:16:43,360
actually, you know what, I need 
to get a GPU. 

1355
01:16:43,360 --> 01:16:46,240
So then I Start learning about 
hosting GPU's and this and that.

1356
01:16:46,240 --> 01:16:48,840
And so then there's this whole 
side quest that I go on. 

1357
01:16:49,160 --> 01:16:53,280
And afterwards I started out 
just wanting to build like an 

1358
01:16:53,280 --> 01:16:56,360
RSS reader. 
I ended up, you know, an expert 

1359
01:16:56,360 --> 01:17:01,280
in Kubernetes type thing and. 
So that's awesome, but that 

1360
01:17:01,280 --> 01:17:05,080
that's almost like I think that 
it's almost like, you know, if 

1361
01:17:05,080 --> 01:17:09,960
you're doing your main thing, 
then if you recognize the side 

1362
01:17:09,960 --> 01:17:13,480
quests of what it is, that's 
very different to deluding 

1363
01:17:13,480 --> 01:17:15,600
yourself to saying that side 
quests actually helping a main 

1364
01:17:15,600 --> 01:17:19,600
quest, right. 
Like I wipe good stuff on the 

1365
01:17:19,600 --> 01:17:21,840
weekend. 
Like I've, you know, like the 

1366
01:17:21,880 --> 01:17:24,280
Asian. 
Like I because a big part of it,

1367
01:17:24,360 --> 01:17:27,520
I mean, I tell myself is because
I want to be in touch with agent

1368
01:17:27,520 --> 01:17:28,760
and stuff, but like, who am I 
kidding? 

1369
01:17:28,760 --> 01:17:31,000
Like I enjoy it. 
Like I, I like it, but you know,

1370
01:17:31,000 --> 01:17:35,520
I want to, but I I just don't 
want to tell myself that Oh 

1371
01:17:35,520 --> 01:17:38,360
yeah, I did all this stuff and 
like, that's definitely pushing 

1372
01:17:38,360 --> 01:17:40,360
Turkey. 
Like it's clearly not. 

1373
01:17:40,840 --> 01:17:42,880
Right. 
It's it's just a hobby. 

1374
01:17:42,880 --> 01:17:45,080
It's another hobby. 
And obviously awesome. 

1375
01:17:45,080 --> 01:17:49,040
Like you learn so much, like 
right, like I you and you never 

1376
01:17:49,040 --> 01:17:51,760
know what you don't know you. 
You're trying new stuff and then

1377
01:17:51,760 --> 01:17:54,800
you learn something. 
I'll tell you what man, there is

1378
01:17:54,800 --> 01:17:58,040
no better feeling, no bigger 
dopamine hit. 

1379
01:17:58,240 --> 01:18:01,680
Yeah, when people talk about 
social media giving you dopamine

1380
01:18:02,680 --> 01:18:04,360
vibe coding. 
Oh, I saw that that was. 

1381
01:18:04,480 --> 01:18:08,280
A good post yes and and getting 
an open source and basically 

1382
01:18:08,520 --> 01:18:14,160
vibe coding a bug fix and then 
getting your PR merged to main. 

1383
01:18:14,720 --> 01:18:16,120
That's some dopamine right 
there. 

1384
01:18:16,120 --> 01:18:20,200
That is just I don't miss that. 
That's our new slot machine. 

1385
01:18:20,680 --> 01:18:24,040
That is what we've got. 
I think that's part of like why 

1386
01:18:24,040 --> 01:18:27,560
it's so addictive, right, 
Because it feels so productive 

1387
01:18:28,120 --> 01:18:32,800
and there's nothing like I I've 
always like this is like total 

1388
01:18:32,800 --> 01:18:37,920
tangent, but like some of the 
more popular YouTube videos are,

1389
01:18:37,920 --> 01:18:42,240
you know, people who go and like
mow the lawn, you know, like 

1390
01:18:42,840 --> 01:18:45,000
people have like you. 
Just have this need to just see 

1391
01:18:45,000 --> 01:18:48,280
something satisfying. 
Like something gets schemed up 

1392
01:18:48,280 --> 01:18:52,560
dude I. 
Actually got into really deep 

1393
01:18:52,560 --> 01:18:57,120
cleaning of my car because of 
too many damn TikTok videos and 

1394
01:18:57,120 --> 01:19:01,320
so I would break out a 
toothbrush and get in the all of

1395
01:19:01,320 --> 01:19:02,440
the cracks. 
And crevices? 

1396
01:19:02,440 --> 01:19:03,800
And did you take a before and 
after? 

1397
01:19:03,800 --> 01:19:09,560
No, I wouldn't film myself, but 
I just the oddly satisfying part

1398
01:19:09,560 --> 01:19:13,680
of watching someone do it. 
I just got, I got influenced, 

1399
01:19:13,680 --> 01:19:19,560
you know, to do it myself. 
And so now my hobby has become 

1400
01:19:19,920 --> 01:19:24,240
deep clean in the car, which we 
make a day out of it. 

1401
01:19:24,240 --> 01:19:28,120
I bring my kids and have them 
work on it too, because they're 

1402
01:19:28,120 --> 01:19:30,560
the ones who make it dirty. 
They might as well help clean it

1403
01:19:30,560 --> 01:19:32,760
up. 
Right, but it also sounds like 

1404
01:19:32,760 --> 01:19:35,520
you're deeply in this open 
source repository too with the 

1405
01:19:35,520 --> 01:19:37,200
PR, right? 
Like that's my point. 

1406
01:19:38,000 --> 01:19:39,640
You know. 
Like it's like this thing of 

1407
01:19:39,640 --> 01:19:41,320
like, oh, I fixed it. 
I made it a little bit better. 

1408
01:19:41,360 --> 01:19:43,680
Yeah. 
And so you see that and it's 

1409
01:19:43,680 --> 01:19:45,680
oddly satisfying. 
And then you get that dopamine 

1410
01:19:45,680 --> 01:19:49,480
hit and you go, I helped, I did 
my part.