We had this like wild discussion in the office where I was
talking to one engineer saying, what are you using?
And he's like, oh, I used caffeinate.
And then I think William walked in and he's like, oh, I'm on
amphetamine. Like anybody else listening to
this would be like, the fuck are these guys talking about?
But these are like legit apps that are there to keep their.
It's not a bunch of junkies that are.
Talking about maybe? Where should we start?
Which one should we do first? Why don't we do something with
you? Like I, I think you're pretty
into the harness stuff, right? Let's let's go.
I I think we can go from like the like tech, like the harness,
like how agents are done to the human.
So we'll end on the human. Harness to human.
Honest to human, that's correct. How much harness is too much
harness, and what even do you feel like is needed on top of
the harnesses that you get out-of-the-box?
I guess like how would you define harness?
Like how do you see it because it's so open?
Yeah, I think about harness as all the stuff that you expose to
the agent to make it perform better.
And so this is I kind of recycled the blog post from that
dude Viv from link chain. And it's like all of the
prompts, all of the skills, all the if you have like logs, if
you have the files, file systems, the model itself is
just one thing or the agent itself is one piece of it.
And then you have everything that you add on to the LLM that
helps it perform better. You know, Claude code Codex Eli,
these are harnesses in themselves, right?
And then you'll have, you know, like I think Viv and Harrison,
like they've got their own harness.
I think open claw is pie, if I'm not wrong.
There's a whole bunch of them in there and they're I guess like
different levels, levels of abstractions.
And I'd say like the Claude code codex Eli is probably the the
most serious level of abstraction, right?
Like it's the highest. Why you you, it's just, you
know, clearly it's kind of just giving in instructions, but then
you can build even more on top of that right before you
actually reach the user. So that's why I'm always like
curious like where that like definition is.
So if we're saying the harness is basically everything, are we
saying that? Are we saying it's between like
the user and like the model, like everything in between?
And then what's So in that case, I don't know how much it's too
much like the way I I think about it, it's like the job of
the harness in this case is really to do to do things right.
Like agents really like fast feedback loops and agents have,
I guess what you couldn't say is like this tendency to do really
dumb things as well. So you almost want to protect it
from itself and you wanted to have this fast feedback loop so
when it does do something dumb, it can react quickly and adjust,
right? I know.
Like I know what it needs to do to adjust.
And when you say protect it from itself, you mean protect it from
an agent that is doing the wrong thing or cause I've also, I was
literally just having a conversation at lunch with my
friend Mark about protecting agents, like sub agents from
stepping all over each other. And so there's many ways you can
protect it from itself, whether it's isolation or it's just like
that's true, giving it the right tool at the right time.
That's true. That's true.
So like, so it's like a really, really simple one.
I remember in the early days of Cleric, we were extremely
careful about how we wrote tools and so we didn't give it access.
So you imagine Kubernetes like we didn't actually give it
access to the Cube Cuttle CLI. We wrote custom tools that were
extremely rigid so the agent could just do the thing that we
wanted to do and nothing else. But then, you know, models get
better and then very soon, like, you know, six months later, that
turns out to be a really dumb idea.
But at the at the time, it makes a lot of sense, right?
Because like you're actually preventing and doing more dumb
things. But now, like once you know,
today, these things are just so good at composing multiple tools
to get to the answer at once. And then you get to a point
where another harness is like the thing that's restricting you
from getting to like, you know, better.
That's why the big question that I have is how much harness is
too much harness? And like also how much is, if
you're trying to really map out what you want step by step and
forcing it into one path, there's going to be times that
you want to do that, but then there's other times where you
don't need to. And so by doing it, you're kind
of wasting your time. Right.
Which is why I think it's an evolution, right?
Like, you know, eventually at the current state of the model
and the current state of like whatever harness you're using,
you, whatever you're doing, if you're building an agent, I, I
hope that you're monitoring these things and you're having
these traces to see like why it's screwing up and why it's
not right. So, you know, like we have, we
document failure modes, like every time it does an
investigation, we do a review and we see what did it do Well,
but mostly like we look at, so we, we, we, we take the trace
and we put it through another element we have documented like
labels, right? And then we try to figure out
like, you know, did you spend too much time, you know, like,
for instance, like did we run this log query that just flooded
the context window because we, we went too wide, right?
Or we tried a query that, you know, took six attempts when it
should have just taken 1 because we were just experimenting right
now, the thing that you don't want is to be too restrictive.
But then when you notice that it's constantly doing these
similar mistakes, then that becomes more of a design
question. Like is that something that you
build into the harness so you almost protect it from itself,
like I said, or is this like it's OK, right?
Like it's, it's not, it's not so fundamental that it's going to
screw up because if you try to overfit and you try to fit fix
that, you're going to screw up something you know elsewhere.
And like you said, if it gets that fast feedback loop and it's
making the mistake but it understands it, can it fix it
and correct course correct in flight?
Exactly. So to me, it's almost like a
game that you're playing where the model is just getting better
and better at reasoning and then it's getting a lot more capable
at handling more subjective kind of tasks like this.
Because obviously like what you don't want is to be very
deterministic where it's like 1 size fits all.
Anything beyond like the most basic use cases, like I sorry,
is probably one of the worst ones because it's so dynamic,
it's so complex. You actually want that to be
some play right for the agent. Like mistakes are OK, like 100%
perfect. Like investigation is very
suspicious. Like it means that either you
like, you really overfit to that like one thing, which is like
weird, or it was just so simple that you probably shouldn't, you
know, pay attention. But now if it's like, you know,
like 20%, it's just not getting anywhere.
It's just constantly getting stuck.
Now you have a problem. And then you could then I think
that makes sense for it to be sort of a harness level, you
know, evaluation. And see, maybe the way I would
frame it is that not everything has to be an agent.
Yeah, right. Like maybe an example that I can
think of is, you know, in our in our CICD pipeline, we go, we we
have actually like we try to do a like a security review and a
like a code conventions review, things like that.
That does not need like 100 different tool calls.
It doesn't need to be like very, very complex thing.
It's pretty simple. Effectively you're trying to
see, OK, like what's the DIF that's trying to be shipped?
And then what is some objective metric of what good looks like
that you're passing to this LLM flow?
And then you just want to know is it pass or fail?
And if it's fail, why? That sounds pretty deterministic
to me. It's got like some LLM involved.
And so maybe that's like, you know, so that's where maybe a
good example of where you don't have to overthink things and
have this really fancy harness and really fancy agent probably
like get something off the shelf and like it'll work pretty well.
Whereas like, you know, if you're trying to do, you know,
like a coding agent or like an SRE agent, yeah, you're not
going to get away with something as, you know, simple as that.
Talk to me more about the evolutions that you had to make
as the models got better within Clare.
Like you were saying, you know we we used to do this but now a
a few months later even we realized we don't need to do
that anymore. Yeah, these are always fun when
you talk about like, all the ways you screwed up.
Yeah, right. Instead of talking about like,
oh, look at, look at, look at. I'll say it's done.
I, I think we're like, you know, when we started, I remember like
we had this like big debate. This is like, you know, late
2023 and we went really fancy. I remember talking about an
architecture of like an Ant colony where, you know, you have
a queen that's talking to its workers and its workers are like
very distinct jobs. So, you know, in this case, like
the queen would be the incident commander almost.
And like it would just fire off all these ants to go get data.
And I mean, I think we just spent a little bit too much time
on that. Like it's.
Trying to create that architecture.
Yeah, because like there was clearly a time when, you know,
just just for context, this was GPT 3.54 just came out.
I don't think Anthropic was really seriously on the scene in
the scene at that point. Maybe they were, but like
definitely everybody was like really around the open AI models
and they just weren't very good. I remember I just for fun, I
looked at our git history and I looked at the like the prompts,
like we were screaming, you know, it was like.
All caps. Yeah, like don't do anything but
Jason, like I just please return Jason like do this, do that.
It's like all caps exclamation marks.
I don't know if we like never expletives, but I wouldn't be
surprised. And then, you know, your
coupling that kind of kind of, you know, basic usage of the
model and you have such like, you know, sort of expensive
architecture. Something has to give, right?
So then we rebuilt everything Like it took us like, I don't
know, like took us at least like 3 months to guys realize, OK,
like this is not going to get us anywhere.
And then we made it a, we, we made it really simple.
And I think that was good. You know, we call it a query
agent where it was just like, OK, like can we just have
something very reliable where I give you a question and you
figure out where to go and just answer the question.
So it's not like 10 workers, no supervisor, blah, blah, blah.
It's like a single agent. And surprising like I think
that's that's actually like cloud code was amazing because
it was also actually very simple.
And that was, I'm not saying like we got simple first, but
like we went really Uber complicated and then we went
simple. And I thought that was like
really effective. The, the other piece was like,
obviously like the tools thing that I talked about, right?
Like we, it was almost like a thing of if you know what you,
the problems that you want to solve, if you fix the tools, we
felt that that was a good way of getting this thing to be as
deterministic as possible where we wanted it to be
deterministic. And so then let the, the model
actually figure out like how to, you know, compose its way to the
answer. But I think, you know,
eventually like what, you know, Clark goes to the world did was
say, well, actually the models have gotten good enough that you
don't have to fix, you know, the the tools anymore.
You can just write bash, you can write Python And run that.
And So what we had was basically relaxing that constraint and
then moving right. And this again is like what I'm
hearing is you're saying, oh, we just were giving like the
harness was less deterministic of it.
We weren't so on top of it. Exactly.
But then The thing is, like, you always pay a cost, right?
Because when you have tools that are that deterministic, security
is not a problem because you know off the bat exactly what
the agent can and cannot do. Yeah, the moment you give it
bash in Python now things get scary, right?
And then so we couldn't really make that switch until we had to
just sandbox the shit out of this agent, right?
So the way at least like my mental model has shifted over
the years is just you want to have as much non determinism
inside the box, but you want to make a really strong box.
Uh huh. So I can't like as much as you
can. I can't you you want.
To you guys were some of the first folks that I heard talking
about sandboxing and you were creating environments too, which
is another one that became very popular because you had to
simulate cloud environments to see if the SRE agent was running
right. Yeah, yeah.
Like it's, I think probably the most complex thing that at least
in our domain, is that every environment we would go to is so
different, right. Like, I mean, I use myself as an
analogy. Yeah, I used to work at Gojek
running the ML platform and I'm working on problems like, you
know, how do you make model inferencing for pricing, driver
allocation, uh, you know, as as efficient as possible.
It's all in Kubernetes. But then everything I just said
is going to sound very familiar to someone at Lyft, at Uber,
any, any, any kind of ride hailing food company.
But it's not like I can join them today and I'm going to be
productive. Yeah, because they've got their
own way of doing something like, yeah, the high level problem
statement is the true is, is, is the same, but the way they do it
is so different. And so, you know, if you assume
that I am the agent and I've gotten really good at these
tools like Kubernetes and all the stuff, I'm still going to
suck when I move into Uber, right?
And so we, the way we try to solve for that, I know, like I,
I think you had a really good job with William and Shreya
about this, right? A few What was it like a year
ago? That was probably a little more
than a year ago now. Right.
Yeah. And so this is the primary
problem, like how do you test very quickly across diverse
environments and know that your agents actually up to the task?
And so then like in like it's, it's been like a really
interesting phase. You know, you're talking about
like the evolution of cleric. The investigation agent has just
become the least important part of the stack.
It's almost like the easiest thing to build now, because that
is like not what's actually durable.
It's it's like these environments, like how can you
test and how do you know it's working?
Like that's durable? Like the learning stuff, you
know, we can talk about, it's knowing I was like, monitoring
it and verifying whether it's correct.
This becomes like far more harder and more durable
challenges. Whereas I think, you know, you
can get your cloud code connected up to a bunch of MCPS
and say, hey, go figure out what's wrong.
You'll probably do fine. Yeah.
Because the search problem is much less complex than that.
Is this working like the verification problem right?
Right, right. Like once you go from beyond a
simple kind of task where it's like, you know, if something's
wrong, it's ironic. Like the more, the more serious
the problem. It's almost like there are some
cases of that, but it's actually very simple because it's
serious, because something is down.
So it's quite binary, right? So it's, it's pretty obvious
this is the problem, right? But like the most subtle it
gets, now things get hard, right?
Because now it's like very reducing credit.
So that's where it's like really hard to know.
That's where the verification comes in.
That's where the learning comes in.
Because like you need to actually like look at the
problem through the lens of that company, like their conventions,
the way they do things. Because if you just come in with
just world knowledge, it's going to be really complicated.
Because like the kind that you know, out of maybe 100 different
reasons this could have failed, you have no way of knowing which
ones you should be paying more attention to because they're
more likely to be the cause. Which an engineer who spent ten
years in the company would immediately do, right?
Which at the end of the day is kind of still the search space,
but you have to verify the search in a much different way
than. And I like what you're saying
there, like the it's a huge problem, you know, oh, the
database blew up or, you know, it's normally pretty clear if
it's a very, very if it's Seb one or whatever incident, you're
like we understand. But when it's just something
that you're like, why is latency spiking by 20% here?
What is going on? Those are the worst.
Yeah. And so then you really have to
dig in and you have to verify through that whole the search
space again and have the agent know if it's on the right path.
And so that's been where you're focusing the attention.
Now you've and figuring out the sandboxes, figuring out the
environments, figuring out how to verify.
Yeah, like how do you just relentlessly and ruthlessly keep
getting the accuracy up, and then as you're doing that, you
keep falling into more and more of this subtle territory?
Then you need to build more stuff to make sure that you're
on the right track, right? Because you once you've moved
away from simple, that's where like all this other stuff that
we talked about matters. I think it's almost like that's
why I've, I've started thinking a lot now about just sort of the
human agent interactions now, right?
Because you know, in a world where the agents, you know,
something like 30 to 40% correct.
I actually, I mean, there's a little bit of a detour, but
like, I was, I was talking to somebody like an engineer who
said, I almost like it when the agent's always wrong because I'm
suspicious and I'll check right. Whereas like if it's like mostly
right, I start trusting it. And then at one time it's like,
so, so your Spidey sense is now. And I'm, I'm, I've just seen the
Spidey sense just continuing to get challenged because, you
know, I can, you know, all the stuff I was talking about with
Tarek in the early days, you know, when we're doing these
tools and things like that, you know, what's not that it's like
what just being explicit about is that, you know, we weren't
very effective outside pretty simple kind of problems, right?
And you didn't trust it at all. I imagine exactly like, unless
it was like very obviously the problem and we found it, you,
you, you, you know, you didn't see that trust barrier being.
So that was like our focus for like, you know, it's been like 2
1/2 years of just like trying to get to that point where like,
like, you know, you're just implicitly trusting this thing
and that's been happening. But now it's creating new
problems, right? Because when that happens,
people are now trying to throw you into more and more complex
areas, like they want to like, give you more work.
And now it's getting a lot more subjective on like what's what's
correct and what's not. And then people are worried
about, well, you know, like, how am I going to keep learning if
this thing's getting so much better, right.
And so that's that's that's. Where you're saying the human
aspect comes in, you have folks that are starting to be like
suspicious of it, but also they're recognizing that, well,
if I'm always thrown it at all of my problems, then am I
outsourcing the thinking? That, but also, I think it's
like more of a, this shift that people have gone through, which
is, you know, say, say 12 months ago, people weren't, I don't
know if people were actually that like concerned, like
outside people who are like really deep into AII don't know
if engineers thought, but these things are really good.
And I'm not sure. I, I, I, I heard that more and
more. No, but we're clearly at a point
like, I think people just went, went into Christmas break,
played around with like all this stuff.
And then we're like, holy shit, like things have really changed.
And now you can just see this trajectory that, you know,
whatever it's not good at now it's going to get there.
Like I know it is. And so now all of us are kind of
kind of thinking like, OK, so what's next?
Like what does what, what happens now where I'm not needed
anymore in this one task, right? So imagine like, you know, six
months ago you're talking to cleric, maybe cleric gets 60%
there and now you have to like have two more follow-ups and get
to their answer and then you're done.
But now he's going 65, now it's 707580.
It's just keeping on going. And then you're like, OK, So
what does this mean, right? Yeah, where do I add value?
Yes. And that's what I'm really
interested in. And so, so part of it is what
you said, which is like the learning loop.
But then also I think it's more, you know, like a more
philosophical question, right? Like where sort of am I So in,
in my mental model? It's like, I don't know, I'm
still working on like what the right analogy would be.
But it's almost like all of us are going to become managers.
I don't think we're going to like, you know, if you look at
sort of the, the kind of the online chatter and like this is
even happening in our in our team, people are token maxing.
People are like worrying about how do I make sure my agents
have what they need to get work done?
Yeah. How does it continue through the
night while I sleep? It still is working.
Right. Like I had this, like we had
this like wild discussion in the office where I was talking to
one engineer saying, what are you using?
And he's like, oh, I use caffeinate.
And then I think William walked in and he's like, oh, I'm an
amphetamine. Like anybody else listening to
this would be like, the fuck are these guys talking about?
But these are like legit apps that are there to keep your.
It's not a bunch of junkies that are talking.
About but but yeah, like so so then what?
You know, if you just like double click a little bit beyond
like the sweatshop vibes, it's basically it's like, hey, I have
these workers and I kind of care about making sure that they're
OK. I want to make sure that they're
not going to be interrupted. They've got what they need to
get stuff done. It's pretty kind of managerial
problem. Yeah, right.
It's just the managers don't tend to give humans
amphetamines. No.
And that's. Well, not yeah, at least not all
of them. Like the ones we you, you should
watch out for. Those are the ones.
That exactly. Yeah.
So, but the apart from that, I agree with you wholeheartedly
because when we had the coding agent conference a month back, I
remember one of my friend's bosses who is there like VPN
Jet, a fairly big company. Like he was so excited because
he went to the workshop and he was like hands on shipping code,
like merging PRS super excited. And then when I wrote about that
on LinkedIn, people were like, yeah, well, why wouldn't they
be? Managers have been doing this.
Like what do you think a manager is?
It's almost like a vibe coder. You're managing agents or you're
managing people. It's just people are actually
kind of harder to manage a. 100%.
Exactly. Humans are very messy agents,
less so. You can really grind and give
them the amphetamines and it's all right.
Yeah. Or you can just be.
Yeah, there's there's no need to be too nice to your.
Agents. Yeah, polite.
They don't ask for raises every couple months or more, shares
all of that stuff. Or they don't like complain
about sitting next to somebody that smells bad.
All of that stuff that humans have to deal with, right?
You don't deal with that and. But it's funny to me because
it's, it's like I, I think people have gotten really caught
up with how addictive it is, right?
And I think that's coming from this place of, you know, like, I
think if you just went through and got to, you know,
eventually, like there instead, it's a very stressful job being
a manager, right? Like you're constantly like,
hey, I've got, you know, 10/20/30 people depending on me
and what I say we're going to do.
I've got 30 times like the bandwidth of that week assigned
to this problem that I've said we have to do.
And now you know, like, and this problem goes from CEO to
engineering manager. It's pretty freaking worrying,
right? Like you.
And so like what I'm seeing with the amphetamine and like all
these conversations is like the the Colonel is that same worry
like I I've got all this like bandwidth of tokens.
Am I spending it correctly? Like am I doing the right thing?
Some of us aren't, but just like shipping all kinds of shit, but
like generally like it's it's just going to end up there,
right. And and then you're back to
being this like person who's like, OK, like here's I've got X
resources. I've got a problem to sort out.
How do I, you know, corral my army of people or like agents to
actually go, you know, solve this problem?
Yeah, make sure we're all rowing in the right direction and
taking advantage of every moment because any moment that the
agents aren't right, looping well.
Hopefully we don't get too stressed out about that because
that, that is a like a dark hole.
Yeah, I think. But yeah, exactly.
You don't have 8 agents constantly running and you feel
like you're what are you even doing right?
I know. I mean, I feel that like I, I,
I've, I don't know the last time why I haven't.
So, so we use, I'm sure like people are using their own
terms, but like internally we use the term like the hopper.
It's like you're cooking like, well, I'm just going to put
another one on the hopper. And so I, I, you know, I, I
need, I feel like I have this need to put something on the
hopper before I go to bed. And I have just infinite
satisfaction waking up to see it's like almost done.
And they're pretty cool with, with the harness discussion and
all that stuff as well, because like you're what you're
implicitly doing is every time it screws up, you're kind of
thinking, oh, what should I have given it?
Like, what could I have given this?
Like, you know, and it's like really what managers do, right?
Like, you know, I was, I was reading a post from, I can't
remember who, but she was like the first thing she asked on her
one-on-one was like, well, how, what's your work set up like?
And can I fix something like, are you comfortable?
I mean, are we literally doing that with agents now?
That's the harness. That is the harness exactly.
What I've been seeing, and I imagine this is going to be
something that maybe it's fleeting, maybe it becomes more
prevalent, is that you have to do a lot of the upfront work in
order to be able to put something on the hopper and let
it run. Yes, yes, but by a print work I
assume you mean like, you know, speaking it out properly.
The scaffolding around it, you can't just kind of like Willy
nilly throw in one or two sentences and be like go and
have fun. Well, you can, but then you're
just going to have a look at pretty shitty.
Shitty outcome. Exactly.
I mean, that's where I guess, like cost and things like that
come in. But yeah, So what I'm thinking
is like, OK, so right now we're kind of like, you know, some of
us more than others are using this more and more like
effectively. And then, you know, what are
these people doing that's more effective and how are we
learning from them? I'm like, where does this all
go? Like to your point, you know,
this whole spectrum and development and the Ralph Wigan
loop and all the stuff like it's coming up because like, you
know, these are like decks was amazing at the coding engine as
you always like. It's all like these lessons
learned from people who are just like doing it all the time.
And we're all trying to learn from it.
But I guess like my, my, my thought here is that I think
it's kind of like you want to be a mix of a manager and a
Craftsman, right? Because, you know, in a world
where you can build anything you want effectively, what is left
is your taste and your judgement, right?
And that's why I think there's a very dark way of taking this,
which is that you lose all agency.
You stop learning like you like this thing is just gonna and I,
I don't know, I guess like if you take any anything from this
conversation, I hope that we can have a more positive view of it,
which is that like actually the really fun stuff about building
product is putting your personality and like your
thoughts and like your view of like what you want to bring to
the world out there. Now you can get there much
faster. And so like, how do we build for
a world like that where that becomes like all the why you
spend all your time on, whereas like the sort of the toil of
getting there is something with the agent is doing.
It goes back to those feedback loops you were saying before.
That's what I was thinking. Like if you know, if you,
because the thing with I think the managerial word, which I
don't like as much is that it almost suggests that you don't
know much about the craft. You're just focusing on the sort
of public mastering aspect of it, which I mean, I, that's why
I don't like the word. I like the whole craftsmanship
thing because like it, it, it's about mastery, right?
So you're almost like in charge of a workshop and you're trying
to get something done. And what you don't want is to
judge based on what's produced. You actually want to know what's
going on, right? You, you don't want to be
abstracted away too much. So I think like, even with the
agents, I think the mistake would be to say this thing's so
good. So we're just going to wait for
it to finish all of its code and then I'm going to review all of
its code. I'm not sure that that's
effective. Well, you didn't want to say
something about and before we hit record about how you want to
be able to see as it's making its decisions, what it's making
and then steer it. And that's steerability I think
is a little bit lacking sometimes.
But also, if you're going to try and scale to running 8 parallel
agents, you just can't expect to steer 8 agents in parallel.
That's true. But that's why I think the point
of abstraction is like useful, right?
Because you know, if you've got 8 agents and you're writing
some, you know, big project, it's going to produce just reams
and reams of code. You can't read all that code.
But an abstraction above that is what are the decisions that it
took to get to that code, right? Like a really dumb example is,
you know, an agent is probably gonna be really good at writing
a Mac OS desktop app. It's gonna get, like, really
good. It's gonna look beautiful.
But maybe the judgement is that we shouldn't write a desktop
app. It should be a web app, right?
Because I want more customers. And like, I, I think there's a
bunch of people in Linux who's gonna like this stuff as well.
I mean, that's a judgement call. And so then there's a deletion
trace somewhere where your agent or your product manager agent is
like, we should build a Mac App. And so then, you know, at that
point you could probably go, well, no, we shouldn't.
I think we should build X instead, so that is, I think,
infinitely easier to review than all the code that was written,
right? But isn't that just making you
the bottleneck? Well, that's the thing, like I
think it if you become the bottleneck, then it should be
for a good reason. You are actually like putting,
inserting yourself in there. I think there's the kind of two
constraints like 1 is your test. So hopefully, like, you know,
you are operating at that level where you're injecting a test
and that ideally is like something you're exercising and
you're getting better at. And the second one is like, I
think the real world also has constraints, right?
Like in a world where everything gets amazing and like you can
produce infinite code, your customers can't take that much
change anyway, right? Like you can't have, you know,
like a lot of these companies and the services that we are
used to took decades to evolve into what they are.
Like, you know, if I, if you just speed rang someone who uses
Uber today to everything that's they've offered, you know, over
the last 10-15 years, I mean, that's not possible that people
can't take it. So I think, yeah, it is a
bottleneck. But like it's they only make
sense if it's adding value. And I think there's those are
two ways of adding value. Well, that's a fascinating topic
to think about. Is where you need the human to
plug in. Because if it is, then going
back to your example of the desktop app versus a web app,
that's one question that potentially you plug in a human
for or you don't, and it's because maybe it just doesn't
matter and you can build it. But like, what decisions need to
actually be outsourced to the human?
You know, where do you plug the human in?
And how does the model know that this is a really big decision?
Before I make this decision and go down this potential path, I
should probably ask someone. Yeah, I, I mean, I think it's,
it's useful thinking about like what the ends did.
And I mean, there's no true end state, but at least like I think
in the next 12 months, like what where I think we should be.
It feels to me like you're kind of approving exceptions, right?
Like when you know, like say you have an employee and they're
supposed to like, you know, do something like, for instance,
like book this podcast studio, right?
It's, it's, it's likely that you're giving them some
guidelines, right? Like here's your budget.
Here's kind of the area that I want you to look at and you
know, you go figure it out. But if you're going to violate
any of these bounds, and there's a good reason, then I need to
know and I I think it's kind of similar here.
But again, then you have to really upfront define those
bounds clearly and put that scaffolding in place and then
let it run. But but I think that's the work
which is valuable. Yeah, right.
Because I think those are the decisions which today, like, you
know, live in people's heads is all implicit decision making
that's happening, right? Which is not explicit, but it,
it was OK. Or at least it's OK in a world
of humans. Because when we are, when we're
not sure, you call a meeting, you send a slack or things like
that, but it just does not work with agents because agents work
on agent time, right? Like you, it needs an answer
right now. Like from an SRE standpoint,
like, well, you know, that's a domain, you know, I think about
a lot. It's like how can you reliably
get an agent to fix something at 2:00 AM if it doesn't have clear
bounds of what it can do and what it can't?
It just can't. It can't call a meeting at 2:00
AM like we call meetings at 2:00 AM because it's an incident and
like you call everybody and was like, guys, what do we do?
Like should we restart this? Should we stop it?
Should we do this? Should we do that?
Unless those decisions become, you know, preserved or like, you
know, queryable, you're not going to get anywhere.
So I think the phase where in now it's all about actually, you
know, I think, you know, Jaya from Foundation Capital did an
amazing piece called Context graphs and which really just
blew up. But I think it's like all around
kind of a similar idea here where that's the valuable data
now. Like what are the decisions that
you know, are OK? Like what are the bounds?
And then how do you actually track the decisions that the
agents are making so that you're actually having this compounding
loop of preserving? And it's these instructions in a
way that humans can read them and, and like, go, yeah, yeah,
that looks fine. No, no, don't do this.
Or at least you're defining an exception.
Like, as long as you're over here, do whatever you want.
But if you're coming out, talk to me because I need to tell you
what to do. Are you guys doing that?
Is that one of those more complex things that you started
to add where you have decisions being made in Slack and you need
to then incorporate that into the agent so that if it
encounters this problem again in the future, it kind of has a
road map? We're doing that.
So that to me is like part of learning where you I don't think
it's as advanced as I'd like, but that's the path which I want
to be on. So you know, I've been railing
on, you know, on social media about like this for deployed
engineer kind of model. The reason I'm not bullish on
that is that you're actually not building the thing I'm talking
about, right? Like what I want when you deploy
cleric is I'm giving it every opportunity to learn these
rules. And if you don't give it that
opportunity and you're just going to throw engineers at the
problem and try to like tune it, then you're not solving that
real valuable problem, right? So to your point on Slack,
there's a wealth of information, like when we get added to
another channel, I mean, there's a ton of stuff like, you know,
when this thing goes, goes down, usually it's because of this
other thing. And here's what, you know, over
the last six months, our engineers have tried and like
this is what generally works. This is what doesn't you can
build a pretty effective map just from this data.
It's, you know, it's like unstructured data.
It's like it's, it's, it's very valuable, right?
And then that can be replayed back to the human to say, here's
what I learned. This is look right, right.
And then now to a human that's actually like a very compressed
form of knowledge, like these are sort of what I've learned,
like what I should do, how your environment works.
Does it look right? And to them it's like very easy
to just edit it. Yeah, I do like the distinction
too of how you're saying the real value is in that type of
thing and being able to create these paper trails in a way or
create that like decision making process the Sops and then doing
the upfront work of creating those so that the agent can do
the hard work of like actually implementing it, Yes.
Exactly. And you, you want to design your
systems to make that hard work like valued, right?
Like, like it, it has to happen like we've tried a lot of times
like to do like in the SRO world.
You'd insist on teams doing postmortems, documenting them,
things like that. It's really hard because you're
asking people to do something with no immediate tangible
value. Like unless you've really got a
good process down. And this happens in every
industry, right? Like just document your shit.
Like, just make sure you're writing stuff down.
Everybody knows that it's a good idea, but it's not necessarily.
But I think agents are super interesting because now you do
get that feedback very quickly, right?
Like if I write something down, oh, I tell the agent, hey,
remember this. And then the next time something
happens, it's like, oh, yeah, thanks, Dee, you told me about
this. So I, you know, I did it this
way instead. Now you feel, oh, OK, that's
that's cool. Right, like so in a way, what
you're constantly doing is updating documentation and
updating. Pretty much like just for the
agent. Yeah, just for the agent and,
and I remember, I think William told me, yeah, normally you
don't necessarily need to ingest the whole Slack thread because
by the end of a Slack thread, there's a decision that's made.
And did you do it with the is it the user experience that like at
the end of the Slack thread, once you know what needs to
happen, then you just like at Cleric and you know, like update
this or here's the information. Or is Cleric just there reading
everything? Because I'm wondering how you
don't have a lot of noise. Well, that's a great question.
Noise is inevitable. You will learn completely
useless things. This will happen, but I think to
us like it's more about curation, right?
Like how do you like, we call it rot?
I think it's pretty standard term at this point that you're
just going to have this sort of memories that you've learned
which aren't very helpful. I mean, there's a couple of
ways. Like 1 is we give people an
explicit way to create them and forget them.
So if it's like we totally screwed it up, we learned
something we shouldn't have, or like we're somehow not learning
something we should, then you can just take over the wheel and
be like, yo, you need to know this.
Like remember so. It's like, yeah, commit this to
memory, yeah. Or forget this exactly
eternally. Exactly.
Yeah, I don't do not mention this to me again.
But like that's usually for extremes, right, Because people
are lazy. Like you got a design for that,
Like you, you don't want to make people say remember this all the
time. So we do implicitly learn as
much as we can like from people's conversations.
We've also tried to make it really cheap for people to, you
know, say goodbye cleric and bad boy like with ratings and things
like that. Because then that also gives a
signal that all the decisions that we made so far, if it's a 5
out of five, it's pretty good chance that we've did the right
thing, right? So then we can look through it.
Whereas if it's like a one out of five, even though we're like,
oh, we did a really good job, then you know, OK, something
went wrong. So either that's something we
missed or like something we, you know, over indexed on or
something like that. So that's that also helps a lot
from the learning. Back, continuing with the noise
theme, I imagine you can and never be satisfied on all the
data that you're collecting like you, whether it comes to all
right, you've got your data dog, you've got your code, you've got
the Slack conversations, you've got the post mortem transcripts,
you've got everything and anything that you can get your
hands on. Are there, is there a little bit
of like a 8020 principle where you say, you know, what if we
just get the Slack conversation and the commit history, that
gets us pretty far. Yeah, I'd say Slack's amazing,
code's amazing, and logs are amazing.
These three get you very far. Slack mostly for just historical
stuff. And this is again, like with the
assumption that you haven't been very disciplined and storing
everything. Like, you know, there are some
folks we work with who've that's done a really good job.
So, you know, if you go to that confluence page, it's like
really well laid out, like all their lessons learned are in a
separate database. So then that's like obviously
much better. But generally like, yeah, like
this kind of text based knowledge is helpful.
The code is interesting because you're, you're looking for kind
of like a source of truth kind of knowledge, because no matter
what you see on Slack or documents or things like that,
you never really know how up to date it is, right?
Like, you know, I, I could say something today and a commit
tonight could completely invalidate that, right?
Like completely change everything because we've done a
big refactor. And so you still need to have
that source of truth. So it's like, do me to us like
code and Kubernetes, like infrastate is really helpful
because the agent can just be like, OK, hang on.
Like, like what is true today? And then like, can I work
backwards to, OK, what do I know historically?
And then like, how do I you, you can't go the other way around.
Like you're trying to piece together a truth from history.
It's hard, yeah. Now, the thing that William
always tells me when I talk to him, it's like it's pretty easy
to get something up and running in terms of an SRE agent that
you can demo. Yeah, like all 100% any any
agents basically. But I get it overnight and then,
yeah, look, look pretty convincing.
And then he's like, but dude, when you get into real
production systems, it is so much harder to get an SRE agent
going. Can you explain like why is
that? What's the difference there?
That is the production gap that you see.
I'd say it's two things. One is just a human element.
The expectations are so much higher, right?
Because I mean, the way I like to think about it is if it's a
coding agent and I got 70% of the code right as a user, you
know, you just save me 70% of the keystrokes I would have to
make. I'm, I'm pretty happy, right?
Like I can, you know, look through it, even if it's not
right, it's, it's kind of cheap for me to know that it's not
correct. I can write some tests.
I can, I can get it over the line.
I'm not, I'm not pissed off with you for being 70%, but we're on
the other extreme, right? Like, you know, if I'm 70%
correct and I LED you down this path, which is not exactly
right, and you've spent 20-30 minutes in a pretty stressful
situation, sometimes you're going to hate me, right?
Like like forget about tech. It's just from a user point of
view like that's. Yeah, right.
Like why did you lie to me? Right.
And you know, and. That's actually, yeah.
I like to think about it. Yeah.
You know, like the, like the
personality of people who are in the, you know, the, the, the
like, the, you know, it's like the, the beast of the belly,
right? Like you're just the belly of
the beast belly. You, you, you get a certain kind
of personality, right? Like you're not going to
tolerate, like you're very risk averse.
Yeah. Adrenaline's pumping.
Right. Because you know, like basically
the kind of companies we work with are larger companies
because like if you're a startup, you don't really have a
production problem, like you don't have that many customers.
But once you have a lot of customers, you're going to
allocate a certain set of people to look after them.
And these people are like, obviously for very good reason,
like very, very careful. But on the then when you bring
into the technical side, that's also where the complexity is
because you know, for I'll just take code again because I think
most folks probably listen to this, are using coding agents.
Most of the truth that you need is in your code, right?
Like like for for a coding agent, you're actually going
deep. You give it this one thing,
you're coding repository, maybe it's couple and you're saying go
build me this new feature. The complexity of the coding
agent is the kind of inputs of the tasks you give the coding
agent is potentially infinite. Like you can ask it to build
anything you want, but the base it's doing it on is finite,
right? It's just with SRE, it's the
opposite, right? The the answer could be anywhere
so that there's no base like it's a, it's a, it's a
horizontal kind of breadth first search.
But the kind of problems are kind of finite, like it's a
latency spike. It's a, you know, something's
down, the user's upset there's a bug.
It's not like, so the complexity really is like from a technical
standpoint that like the, it's a needle in the air stack, like it
could be anywhere. And so when you couple that with
the user problem, hopefully that gives you a sense of why this
is. Yeah, flip that right on its
head, dude. Oh, man.
One of the big questions that I am sure you think about a lot is
like, are we just going to end up becoming a skill?
Is this whole thing going to get so good that now we're a
markdown file that's part of a coding agent?
You don't like my default mode is I'm just a very optimistic
guy. So that's probably like a good
thing to just put it out there. Like that's why I think Willem
and I work really well because he's definitely a lot more like
on the like he'll see all the risks, like you know him well,
right? So I see all the optimism and
then we're trying to like meet in the middle.
I think that's part of why we should be having this this
discussion and like people like I should be talking about this
more because like the whole point is to avoid that future,
right? Like that's why I think, you
know, redefining how you see yourself as this Craftsman, as
this manager and like, where are you actually getting value?
And just knowing where the value is and just going for that.
That's what you got to do, right?
Because you know, you know, you just think about like there's 2
problems that obviously there's many, but I'm trying to reduce
it to 2 problems that the AI world is working on.
The first one is like, can you produce output that's, you know,
like problem free, like no bugs. It's you know, what I'm saying
is generally correct or rather like it, it works.
And the other one is like, is the work you're doing correct in
that? Like is it directionally what,
what you want? Like is it a, should I build a
Mac OS app or should I build a web app?
Like that takes subjectivity and taste.
Whereas the first one is more like, you know, if I asked you
to do a thing, did you do it right?
If your whole world is around the first problem, I mean, I've,
there's, I don't have good news for you.
Like, like it is you have to assume that AI will just get so
good at solving that first problem that you need to think
about like, OK, what happens with the second problem, which
is like, you know, that's where the subjectivity and taste piece
comes in and the real world constraints that you can't just
build everything overnight. People can't take it.
Then I think you want to be like, how do I be the purveyor
of taste and how do I be the one who's really good at, like
making these decisions? Because I have this thing that
can actually produce working artifacts, but how do I make
sure it's better correct working ones?
And continuing along those lines, do you ever feel like
Cleric could just be abstracted into a skill?
Could be. I mean I, I think.
Like. I think our job as founders is
like to constantly think about what is a durable problem and
what is a problem that you should not be spending an iota
of time on, right? Like, like when we started my,
my big rule to everybody was like, we are not fine tuning
models. Like this is like ludicrous.
People will figure this out. They're going to get way better
at like building way better models.
Just assume the models will get better.
I know you hate screaming at the model with JS, like please give
me JS on, but this will be fixed.
Don't worry. Like spend time on the agent.
And then our whole thing like last year was like the agent
layers commoditized. Like it's a waste of time.
Like I think there's going to be way better harnesses than we can
ever build. There's going to be like better
agents and all that stuff. So then, you know, what we're
thinking about is like very much on, you know, the verifiability
and learning and things like that.
I think the same time next year I'm going to be saying something
different. Like that's the point, right?
Like you don't build something that everybody else is going to
do better than you. Like build a thing that's hard
in your domain that's going to last.
And like, I think the really tricky thing with AI is that
that thing is changing every 12 months, but then you got to be
primed to keep making that jump every time.
You will talk to me about what 2027 looks like it you kind of
mapped that out in this blog post that you wrote.
Right. I was trying to think about like
what this world looks like. And to me it's almost like, you
know, you're, you're waking up and you've, you've had all these
agents that have been working, you know, for you and then
they're sort of surfacing exceptions.
So maybe your coding agent shipped a change like to a
config and something ended up breaking at 2:00 AM and your SRE
agent notices this reverses the change and watches what happened
until the metrics seem to be fine.
And then you when you're waking up, you're just getting a report
that you know, this is what happened.
Here's the decisions I took. Like, you know, I noticed that
the coding agent did X, so I decided that this was wrong, so
I reversed it. And so, you know, hope that's
OK, Don't worry like everything's fine.
Or it could be like it's about to make a mistake because it
doesn't know. So it's like surfacing kind of
like a question to you, like should I roll back?
Because I don't know, this is uncharted territory.
But that's almost like to me, the interface that we will be
operating with these agents. So to me, there's kind of two
interesting parts there. One is I call it like the
one-on-one problem, which is like as a human, like at what
level am I doing my one on ones with my agents?
And that's like the policies that we're setting, right?
So you know, like it's OK to roll back if X happens.
This is the way we build things, things like that.
And then the other one, I call it like the stand up problem,
which is I think it's going to get up maybe a little bit more
interesting where once you have teams of agents, I'm not sure
we've done a great job yet as an industry to have these agents
working together, right? Like today, you know, Cleric's
an SRE agent. And I've just talked about how
it's totally different to the coding agent, but it's so
obvious that we have to work really well together.
Yeah, and how do you share context?
How do you share context things like?
That how do you work off of the same planning, right And you're
able to cross one thing off or yeah, that's that is a really
interesting one with with the proper isolation, right.
So it's a little bit tricky in that regard because you want it
to be isolated, but you want it to also share and share
resources when needed. And so it gets, yeah, messy.
I mean, I, I think, I think what's been really interesting
about just thinking about agents is that there's so much overlap
with like how we've solved the same problem for humans.
And like, you can borrow a lot of that stuff.
Like, I don't think we need to reinvent the wheel in this case.
You know, you forget about agents.
Like I've got a product engineer.
A product engineer's main motive is to increase the value of the
product to their customer. They're not.
I mean, hopefully they are, but they generally don't think about
security and reliability. They're just like, ship this as
fast as I can get my promotion. And it's always like, whoa, what
are you doing there? Slow down like this.
This could break something and that's going to be bad.
Security will have the same thing, right?
But then like, how do they work together?
So, you know, there'll be something like if it does a big
change, maybe the product engineer will be like, hey, I'm
shipping APR. Can one of you guys have a look
and let me know what you think? Pretty sure it's OK, but you
know, let me know. There'll probably be some
conventions that the SRE has written, like here's how you do
incidents, here's how you investigate.
Yeah, I was going to say that that feels very synchronous.
And there's probably a world where it's happening in parallel
or it's they're happening together or yeah, like
conventions might be a very rudimentary way of looking at
that. Or they're just one is writing
the code and the next thing you know, there's not even like APR
that needs to happen because one writes the code, the other one
comes and refractors it as it's happening, right?
But I think like, at least on the human side, it's more the
recognition that you have these two entities who are experts in
their own domain. And we've found very slow ways
for them to talk to each other. But like we, we agree that these
are two separate entities because they're very separate
objectives. So they they do belong in the
org, but you know, they've got to talk.
So I don't, I just think that, you know, by 2027 we'd
definitely, whether I figure this out, like I know, like
we're thinking about this deeply, like how do we integrate
even better with, you know, the cloud codes and the codexors of
the world? And how do we actually bring
back all the knowledge that we've gained from production so
that, you know, you're giving it to the coding agent.
So. The coding agent doesn't make
the same mistake exactly. Yeah, that to me is value,
right. Like that's, you know, like one
of the things which is like my big bet beeve is, you know, the
whole point of SRE is you're trying to protect the
reliability for your customers. It's not to reduce incidence.
Like that's a byproduct, right? Like the whole point is like
that's your main goal. So if you can prevent the
incident, that's what you should be doing, right?
So seeing like any model where it's like, oh, I improve MPTR
and I've like I'm on jumping on every incident my F.
One score, it's just like the back in the day with the machine
learning models and being like, wow, my F1 score's amazing.
And then you're like, actually nobody uses that model, so.
Where you missed the prime. Yeah, exactly.
Exactly. Oh man, I did want to ask you
about how you're bringing the learnings back from production
into the coding agents. So I, I mean, I, I think it's
rudimentary right now. So today you can have the coding
agent speak to Cleric through MCP and, you know, you kind of
like get Cleric to weigh in on what it's on, what he thinks.
We also have, you know, like these conventions that you can
have in the repository, which we know that they're coding agents
also use things like that. But I don't think that that's
where it's going to end. I think we, you know, like
there's already kind of attempts.
I know that Google's working on A to a I'm not sure how much
like the adoption has been, but like you, we're trying to create
these open stands for these agents to talk to each other,
right? We haven't adopted any of these
yet, just to be upfront, but I think they will become more and
more of a thing because to your point, you don't want it to be
synchronous, you want it to be like info.
Yeah. And how do you just keep it so
that it's not blowing up the context when you do have
whatever it is that you want to give it as much information
about what you've learned from production, but you don't
necessarily want to give it everything because then you're
just blown up the context every time.
And so is it where you need to even add it to the context or
you just have that sub agent that's coming in behind every
coding agent that is your sub agent, SRE agent that says, Oh,
yeah, I know about these things. And I know that this happened
recently and it has all of the necessary information.
And like, that also doesn't feel like the right way to do it, you
know, so have some clean up agent that is constantly going
behind the coding agent. And it's just like, but how do
you make sure that you do get that 360 degree?
You have the security, you have the, the stability, you have all
of these things that you're checking for.
I, I've heard when we had Sid do the coding agent conference, he
said that one thing that they'll have is an adversarial agent
that will try and just, you know, find problems with
whatever it is. And so that's one way that
they'll do it, but really relaying the information from
production to recognize that, hey, this happened.
It's the second time that this has happened.
Let's not have a third time, right, guys?
Come on, make sure. I just, I want to see what the
dynamics are of the agents that you wake up to.
And the SRE agent is just Debbie Downer all the time.
Like you know this PM the agent is constantly shipping crap and
I have to clean up after. What is this?
Wait, you don't like that's happening now And it's already
just, it's, it's hilarious. Like we, we have a sub agent
that we call the investigation auditor and the investigation
auditor's job is to be Debbie Downer.
It's like you have not done a good enough job.
Keep going. And he is like, why he's screwed
up and things like that. And then, you know, sometimes
the main agent will complain saying like this is like too
many Times Now I, I think I've done the best I've heard.
So I'm just going to return back to the user.
Yeah, it's, it's, it's really funny just reading through these
traces. But it's not even mentioned that
because I think, you know, like stuff I talked about at the
beginning, like, you know, the Ant colony and things like that.
Like, I think we've got to the point where we're now doing a
lot of experimentation on what are the right abstractions,
right? Because I think, you know, for a
long time it's been sort of the main agent, only then it was
main agent, sub agent. Like I think and topics are
releasing. I think it's still in beta where
you actually have like Asian teams, which are some totally
different agents and they don't share any context and they're
just like talking to each other through, you know, some form of
protocol. So I think we'll just have more
and more of these things. I think, I think to me like the
coding agent, SRE agent, the team of agents paradigm makes
much more sense because, you know, coding agent doesn't need
to know the internals of the SRE agent.
The coding agent just need to know that, hey, that's his
expert that I need to talk to you about.
Yeah. And you know, but you know, I'm
not, I'm not going to totally listen to this agent, but I just
want to get some, you know, like, isn't that like a kind of
a consulting thing where it's like the racing matrix, like
responsible, accountable, consulted, informed.
I think you'll have that for agents, right?
Like, you know, you must listen to the SRV agent or no, you may
consult the SRV agent and things like that.
So I think these kind of like governance structures that we
come up with are going to be very, very valuable.
Yeah, a lot of the software that we use and love is going local
1st. So as opposed to me building
something and exposing it to the world, I build something that I
exposed to myself. And so because of that, I don't
necessarily need to have all the security and all of the sure, it
doesn't need to be battle hardened for production if it's
just me as the main consumer because it's very bespoke
software. And now, even more so, I'm
looking at it like, wow, a lot of things that I used to build,
I don't even need to build anymore because I just create
them as a skill. Sure.
And so it doesn't require a full-fledged app, it just
requires a very clean skill, right?
And so the, the act of moving from production grade software
to skill, which is very, very thin.
And it's like the lightest form of a application possible,
right? It's just markdown, right?
So that's, that's kind of how I look at it.
Not to say that the software is not, but like, I still feel like
there's a lot of very valuable software out there.
And so I'm not fully bought into the software's cooked thing.
There's just a class of software that you just never going to
write code in my view like. Payments processing, Stripe,
you're not gonna. That that's a good one.
But like, you know, some, some like like Big Query, for
example, right? Like we dump a lot of our data
in there just for analysis, things like that.
I'm not gonna wipe code that like, I mean, it's just doesn't
make any sense. Like the this is a product that
probably took years and years and years to get right.
And there's almost like a level of accuracy that you expect.
Like when I write a query, I get it exactly like what I want back
and it's performant and all that stuff.
So, you know, physics still applies, right?
Like there's cost of having an engineer thinking about writing
the software, making sure you're maintaining it because it's,
it's always been true with software that the easy part is
building it. The hard part is maintaining it.
That's the reason we exist. You build a whole bunch of shit,
it's in production now, you got to maintain it.
And now most of the engineers are spending time maintaining it
and not actually getting to create.
So our whole value property is, you know, you can go create now
because we can take some of that work, but the cost is real,
right? But yeah, you're right.
Like there's going to be, you know, but some software where, I
don't know, there's like 20 features, but you only need one.
Maybe that's just the skill. Maybe that's like a little app
that you build. I think where things get more
interesting to me is like in that, in that like kind of
middle ground where I feel, and I'm pushing for this internally
a lot where we want to identify everything we do.
So I mean, I, I remember like like a year ago, William and I
were just really frustrated by how long it took to get a
website update done. It just, it was like, you have
so many ideas, but I was like, you need a, there's a meeting,
you know, talk to this person, that person, blah, blah, blah,
blah, blah. And I think like a few months
ago, like, all right, that's it. We're moving to Astro.
Everything's going to be code. A whole marketing site is
basically an Astro site now. It's no longer on any kind of no
code platform. And man, it's amazing, like.
You're just so much faster. So much faster.
That's wild. It's and so so that's going to
happen, right. Like it's like if your platform
is not agent native in that agents like talking to you and
working with you, then that's that's trouble.
Yeah, right. The part that is worth
considering is again, what's durable?
What are the things that you're not going to vibe code?
Sure, versus what are the things that you could probably get away
with vibe coding? Or what are the things that you
definitely need to be a gentic first or you want to be able to
use with coding agents, You want to be able to have that seamless
experience with coding agents. And so like looking at those
that Spectrum is another good one.
And I've, I feel like the system of records thing that's probably
not going anywhere because that's pretty hard to do
Databases, like you're saying, Big query is another hard one.
Like what you really are going to go vibe code to database.
There's very performing databases and they've been
performing. Tested through like years and
years and years, yeah, I think it's almost like a dimension
thing, right? Which is like, how hard is it to
actually build? And what is the friction it
creates for you to get an agent to work with it?
And then if that friction is high enough and the amount of
time you need to build it. Is.
Oh, that's a nice little diagram, yeah.
Now that's going to be white coated.
I'm sorry, but it's it's gone. You're feeling that frustration
of like, I really want my agent to be able to plug into this.
Yeah. And could I build it?
Let's try. All right, Because I want to be
able to have something that my agent can plug into easily.
If my agent builds it itself then it's definitely going to be
able to plug into it. 100% like, I mean, I at this point, I mean
I've, I've, you know, I have an agent that I've connected to
like all my stuff like my Gmail, my, you know, hotspot and
everything. And I'm constantly looking for
like OK how do I make this thing faster and better?
You know what the biggest one for me was?
My buddy JQ said, hey, we got rid of linear.
We just went to all markdown files.
Our whole thing is all markdown files and agents are able to do
whatever with the markdown and they update it and they keep
state on how the files are going.
And so he's like, we realize like, what are we paying all his
money to linear for? We don't even need it because I
don't he he was saying, I realized I wasn't going into
linear and looking at it and all I was doing is asking the agent,
Hey, what's the status on this linear issue?
And so there was no need to actually have linear.
But then the funniest part and it's full circle moment, he
said, yeah, so now we realize there are people that like to
see the status. So we're now vibe coding a front
end. So I'm like, wait a minute.
So I mean the irony would be if you, I guess that's what you're
implying with going full circle, you're kind of building linear.
Yeah, in a way that is much more lightweight and he doesn't
necessarily need like linear. I thought it was durable, but it
at the end of the day it's not really durable, right.
Like all of your, your status can be marked down and so if
everything is marked down and then it can grab context from
other markdown files. So if you're you really go all
in on a file system, then why do you need linear to be the middle
man? I would disagree that, but like
I I think for everybody it's different.
Like to me, in that same dimensions, linear is very low
friction for an agent to use. I think the we use it
internally. So although I do wish to release
an official CLI, I'm not a big fan of the MCP.
But generally, there's so much training data on linear that
agents are pretty good at using them.
And yeah, the, the effort to wipe coat it is probably not
that high. But since the friction is so
low, you're just like, it's not really worth my time to be
thinking about this thing. I'd rather go focus on something
else, right? So like to me, it's like the
friction has to be really high that it's like actively impeding
me for me to think it's worth going ahead because and to the
point now, am I building a linear alternative or am I
building my business? That's what I'm thinking.
I wonder if it is for JQ. The friction was just in him
seeing the bill. Could be, yeah, it could be
that. Was what he was like, man, what
is this like it it just eats at him.
And so maybe it was just like a a little bit of a side project
that became a a bigger project and he realized wow, all right,
let's yeah, let's just see. And if you look at the bill, I
don't think it's that expensive, right.
So he really probably was like, let's just try.
But he also is trying to get agents doing everything.
Like he has five different names of agents in his slack that
he'll call on to do things. You know, because they're the
different agents that oh, this is the whatever linear knock off
agent, right, right. Linney Linney Yeah, like, but
but actually it's ironic that we're talking about linear
because I think this CEO of linear, I don't I can't remember
his name, Carrie. I think he's probably got one of
like this court that I, I really liked.
He talked about how like a whole point of building a company is
like you got to be focused on your main quest side quest will
keep coming, but you got to focus on your main quest.
Like, that is like your number one thing as a as a founder.
And the thing with the agents is that the side quests have become
really tempting. Yeah.
'Cause you can, you know, you look at like what you're, you're
like, I'm just like reflecting as you know.
Yeah, what JQ did, he was like again.
I don't know, like it could be like a so I can't I can't speak
to like whether there was a side quest or not.
Like it seemed yeah, like I don't want to roast him on like
something I don't know. But like one of the things which
I, I've I've seen is that being an engineer founder is actually
one of the hardest things because like coding feels so
good. You feel like you're being
productive, you're getting shit done.
And when you see stuff like this, you can be like, Oh, I can
help my team. I'll just, you know, build this
thing. But then like to me, it's like,
well, that's not what's going to make Derrick successful.
I I don't think, you know, like my team needs me to do like a so
the only thing like like we've really like wide coated
seriously outside like cleric is like this, like this one off
kind of MCPS or CL is that makes it easier for us to maybe like
respond back to a customer or something like that.
But all the tools that we're building is really either to
improve the CICD or something like, but if it's does a thing,
it's working and it's not crazy expensive science living.
You know, it's the exact opposite of what my talk was in
the coding agent conference, cuz I, my whole talk was called
enjoy the side quest. Right, right, right.
And so it was saying how I gave the example of basically me
being like, oh, so I wanted to build an app and I wanted to
schedule it. So then I Start learning about
like GitHub workflows and boom. So then that's cool.
But then I wanted to make sure that it was using a local model
because it started to get too expensive when I was sure doing
this. So then I grabbed and I, I
started learning about like quantizing models and run it on
my own machine. Then I was like, ah, well,
actually, you know what, I need to get a GPU.
So then I Start learning about hosting GPU's and this and that.
And so then there's this whole side quest that I go on.
And afterwards I started out just wanting to build like an
RSS reader. I ended up, you know, an expert
in Kubernetes type thing and. So that's awesome, but that
that's almost like I think that it's almost like, you know, if
you're doing your main thing, then if you recognize the side
quests of what it is, that's very different to deluding
yourself to saying that side quests actually helping a main
quest, right. Like I wipe good stuff on the
weekend. Like I've, you know, like the
Asian. Like I because a big part of it,
I mean, I tell myself is because I want to be in touch with agent
and stuff, but like, who am I kidding?
Like I enjoy it. Like I, I like it, but you know,
I want to, but I I just don't want to tell myself that Oh
yeah, I did all this stuff and like, that's definitely pushing
Turkey. Like it's clearly not.
Right. It's it's just a hobby.
It's another hobby. And obviously awesome.
Like you learn so much, like right, like I you and you never
know what you don't know you. You're trying new stuff and then
you learn something. I'll tell you what man, there is
no better feeling, no bigger dopamine hit.
Yeah, when people talk about social media giving you dopamine
vibe coding. Oh, I saw that that was.
A good post yes and and getting an open source and basically
vibe coding a bug fix and then getting your PR merged to main.
That's some dopamine right there.
That is just I don't miss that. That's our new slot machine.
That is what we've got. I think that's part of like why
it's so addictive, right, Because it feels so productive
and there's nothing like I I've always like this is like total
tangent, but like some of the more popular YouTube videos are,
you know, people who go and like mow the lawn, you know, like
people have like you. Just have this need to just see
something satisfying. Like something gets schemed up
dude I. Actually got into really deep
cleaning of my car because of too many damn TikTok videos and
so I would break out a toothbrush and get in the all of
the cracks. And crevices?
And did you take a before and after?
No, I wouldn't film myself, but I just the oddly satisfying part
of watching someone do it. I just got, I got influenced,
you know, to do it myself. And so now my hobby has become
deep clean in the car, which we make a day out of it.
I bring my kids and have them work on it too, because they're
the ones who make it dirty. They might as well help clean it
up. Right, but it also sounds like
you're deeply in this open source repository too with the
PR, right? Like that's my point.
You know. Like it's like this thing of
like, oh, I fixed it. I made it a little bit better.
Yeah. And so you see that and it's
oddly satisfying. And then you get that dopamine
hit and you go, I helped, I did my part.