It is March 30th, 2026, and I want you to imagine something
for a second. You're sitting at your desk,
sipping your coffee, and you're reviewing a routine pull request
from a junior developer on your team.
You're scrolling through the code and you know everything
looks totally normal. The logic flows, the syntax is
clean, and then right there, nested in the middle of a
function, you find an advertisement just quietly
slipped into your company's proprietary code base by an AI
assistant. Wow.
I mean, it sounds like a cyberpunk dystopian novel,
honestly, but that is the reality of our current
development environment, yeah. It really is.
For today's deep dives, we're pulling from a really
fascinating stack of sources to make sense of the absolute chaos
in the AI developer tool space over this last month.
We're looking at hacker blogs from Zach Manson and Simon
Willison. We've got some incredibly
detailed reverse engineering tear downs from the Arklot AI
subreddit, official announcements from GitHub and
Anthropic, and some really good insights in the Latent Space
newsletter. Yeah.
And the mission for this deep dive is to really try and
navigate this wild acceleration because right now, the
productivity gains are just massive.
I mean, people are doing things like vibe coding entire
applications in an afternoon. Exactly.
But the hidden traps are equally massive.
We're talking about giant silent API bills and, you know,
shifting data privacy policies that could just leave your code
completely exposed. Yeah.
The risk? So let's unpack this.
The way I see it, the current state of AI coding tools is a
lot like handing the keys to a brand new Ferrari to a teenager.
You're going to get incredible mind bending speed, but you
absolutely need to know where the brakes are and you really
need to know who is tracking the GPS.
That is a perfect analogy because that blind trust in the
GPS is the core issue here. The transition from AI is a fun
novelty to AI is a deeply integrated Co worker.
Happened so fast that human review is often just, well,
nominal. All right, it's a rubber stamp.
Exactly. We see a block of code, it has
the right shape, the right syntax highlighting, and we just
assume it's correct. Which brings us back to that
advertisement in the pull request.
This comes straight from a blog post published just a few days
ago by a developer named Zach Manson over at
notes.zachmanson.com. He reported that GitHub Copilot
actually inserted what appeared to be promotional content
directly into APR. Like how does a coding assistant
even hallucinate a billboard? It's supposed to be writing
functions, not selling me things.
Well, it comes down to training data language basically, and the
fundamental mechanics of how these large language models
predict the next token. Because these models are trained
on millions of public repositories, right?
And a lot of those repositories contain markdown files, sponsor
Readme's, or boilerplate templates that include
promotional links for the creator's Atreon or, you know,
Cororate sponsors. Oh, I see.
O When the AI's context window gets confused, maybe the code
structure closely resembles a popular open source template it
was trained on. It just redicts the next most
likely token, and sometimes that sequence of tokens happens to be
an ad from its training data. So it's just blindly
regurgitating a pattern it saw in the wild without any actual
understanding that an ad doesn't belong inside a Python script.
Precisely. I mean, the incident completely
blew up on Hacker News because it highlights a really dangerous
trust gap. There's a massive difference
between code that usually looks right and code that is actually
right. Yeah, when we get so comfortable
with a tool that we stop actively reading the code it
generates, we just leave ourselves incredibly vulnerable
to these bizarre hallucinations. And it feels like this localized
trust issue, like worrying about what the AI is putting into our
code, connects directly to a much broader corporate level
trust issue regarding what the AI companies are pulling out of
our code. 0 Absolutely. Because the very same week Zach
Manson's post drops, GitHub makes a massive data policy
announcement. Yeah, that GitHub announcement
from March 25th is critical. Listeners, if you take nothing
else away from this deep dive, mark April 2420 2026 on your
calendar. You.
Usually write it down. Starting on that date, the
interaction data from Copilot, Free Pro and Pro Plus users will
be used to train AI models for Microsoft.
And by interaction data they mean everything right?
Like inputs, outputs. Everything your prompts, the
code snippets it generates and the entire context window of the
files you have open. Wow.
And the kicker is that this change is opt out by default.
You have to actively go into your settings and tell them not
to use your data. See, I really have to push back
on the philosophy behind an opt out mechanism here.
I mean, are we entering an era where PR review is just theatre
and if we blink, our proprietary secrets and our core business
logic just become part of the collective AI hive mind?
It's a valid fear. It feels like we're paying a
premium subscription for a tool that is secretly using our own
blueprints to essentially replace us.
If I have proprietary code passing through Copilot, I don't
want to have to remember to uncheck a box buried in a
settings menu just to keep it safe.
Right. It forces us to ask what the
true cost of these tools actually is.
We pay the subscription fee, sure, but we are also
potentially paying with our data.
And as we've seen this month, the frantic pace of development
in this space means we might also be paying literal
unexpected financial costs when these systems breakdown behind
the scenes. Yes, because the speed of
development right now is just blinding if.
They even accidentally leaked the name of something called
Claude Mythos which they quickly scrubbed.
So now everyone is guessing about what massive release is
coming next. But honestly, the real issue is
the regressions. Right, the regressions.
I saw a massive thread on the R Claude AI subreddit where
someone's API bill absolutely melted down, but I couldn't
quite understand how they figured out what went wrong.
They were talking about reverse engineering the app, which seems
incredibly intense for a simple billing dispute.
Oh, the lengths this independent researcher had to go to are
staggering. They were dealing with clawed
codes, stand alone binary, and this isn't some small readable
Python script, it is a 228 megabyte ELF binary.
An ELF binary, so it's compiled. Exactly.
ELF stands for Executable and Linkable Format.
It's basically a package compiled stand alone application
format used in Linux and Unix environments.
Because it's compiled into machine code, it is essentially
a black box. You can't just open it in the
text editor to see how it works. So they actually had to pry the
black box open. They spent days tearing it
apart. They were using heavy duty
reverse engineering tools like Gidra and Raider 2 and they were
setting up man in the middle of proxies just to intercept the
network traffic so they could see what the tool was actually
sending to Anthropic servers under the hood.
That is wild. What did they find hiding in
that 228 megabyte black box? They found two independent bugs
that were silently breaking prompt caching.
Oh, no, yeah. One of them had to do with a
Sentinel replacement issue. OK, wait, what's a Sentinel in
this context? Just so we're on the same page.
Sure. So in programming, a Sentinel is
just a specific unique value used to signal the end of a
block of data. It's like a bookmark telling the
system, hey, stop reading here. Got it.
Because Claude code was mishandling these Sentinels, the
system was losing its place. It thought it was reading a
brand new file every single time, which totally destroyed
the prompt cache. Think about your own workflow
right now. If you're using Claude to
refactor a massive code base prompt caching is what makes it
financially viable. If you send the same 100,000
lines of code to the AI over and over, caching means you only pay
a tiny fraction of the token cost for that repeated context.
Exactly. But because of this Sentinel
bug, the passion was feeling completely The system quietly
defaulted to charging the user for full token prices on every
single interaction instead of those discounted cash hit
prices. And there were 0 error messages,
right? It didn't crash, it just
silently charged them full price.
Heavy users of clawed code. We're seeing A10 to 20X
inflation in their ATI costs silently while they thought
everything was functioning normally.
I mean is the move fast and break things era back?
But instead of just crashing our apps it's silently siphoning our
wallets while we sleep. It's just wild to me that a user
had to fire up Ghidra and decompile machine code just to
figure out why their credit card was melting.
It is a perfect storm, really rapid iteration combined with
complex abstraction. The tools are incredibly
capable, but they are shipping with deeply hidden bugs.
This means we urgently need a way to control them.
We have to learn how to bound the infinite intern.
Bounding the infinite intern. I love that that perfectly
captures the feeling of using an autonomous coding agent.
They're incredibly eager, but they have absolutely no common
sense. Yeah, Simon Willison recently
quoted Matt Webb on this exact philosophy.
Webb said agents grind problems into dust.
What Webb means is that if you give an AI agent a while loop,
in enough time it will technically solve any problem
you give it. Unlike a human intern who will
eventually get tired, realize they're stuck, and ask you for
help, a machine will just keep iterating.
It just doesn't. Stuff it'll try 10,000 dead end
solutions. Bring a trillion tokens in the
process until it either hits a hard rate limit or drains your
bank account. So the challenge isn't getting
the agent to solve the problem eventually.
The challenge is getting it to solve the problem efficiently
before it grinds your wallet into dust.
This is where we get some incredibly authoritative advice
from Boris Journey. He's the creator of Claude Code.
He dropped 15 best practice tips on the R Claude AI subreddit and
GitHub, and they are essential reading.
The core of his advice revolves around mastering the claw G dot
MD file. See, I always thought of claudy
dot MD as just a basic config file you know, where you list a
few project paths or environment variables.
Yeah, a lot of people do. But Cherney emphasizes that you
need to completely change your framing.
You must treat Clodgy dot MD as a comprehensive onboarding
document for a hypothetical brilliant new engineer who
cannot ask you any follow up questions.
Oh wow, that means you have to pre answer every possible
misunderstanding. Exactly.
You need to include your architecture decisions, the
rationale behind them, your testing conventions, deployment
gotchas. You should explicitly list files
that Claude is absolutely not allowed to modify, but the most
genius piece of advice Charney gives I think, is to include a
specific section for things that look like they should be
refactored but shouldn't. Oh, that's brilliant, because AI
loves to be helpful. If it sees a piece of code that
looks redundant or strangely structured, its instinct is to
quote UN quote fix it for you, entirely unaware that you
structured it that way to handle some bizarre undocumented edge
case in a legacy API. Right, you have to explicitly
bound its scope. Cherney also mentions giving
Claude tasks with defined done states.
What does that look like in practice?
Well, don't just prompt it to fix the login page, you tell it.
Add input validation to the login function, stop immediately
when the unit test pass and do not change anything else.
Makes sense. We also need to talk about
system permissions because Claude code recently introduced
auto mode. Oh yeah, Simon Wilson read about
this on his blog. Auto mode seems to be the safer
alternative to using the dangerously skip permissions
flag, which sounds terrifying just reading it out.
Loud it does in auto mode. Claude makes permission
decisions autonomously, like deciding whether it needs to run
a bash script or install a package, but there are hidden
safeguards monitoring his actions.
Wilson cautions, though, that you need to read the
documentation carefully before adopting this, especially if you
are working on a team with access to production
infrastructure. The risk profile of letting an
agent run wild on a solo weekend project on your local machine is
fundamentally different from letting it loose in a shared AWS
environment. It's like handing a bowling ball
to a hyperactive bodybuilder who has never played the game.
They're going to throw it at 100 miles an hour, which is
incredible power. But without Klow, you dot MD and
explicit done states acting as the gutter bumpers, that ball is
going to crash right through the wall of the alley into the
parking lot and rack up a massive bill.
That is exactly the dynamic we are managing.
So we've explored the trust gap, we've unpacked the silent cost
leaks, and we've established how to set up the bumpers.
If you manage all of these risks perfectly, what is the actual
reward? What kind of magic can you build
right now? This is the fun part, because
despite all the chaos and the bugs, the raw capability of
these tools is just staggering. Simon Wilson has been writing
heavily about Vibe Coding. Vibe coding.
It describes this entirely new paradigm where you aren't
writing every line of syntax yourself.
Instead, you're directing the AI at a high level.
You're feeling out the architecture and the flow,
acting more like an orchestrator while the AI handles the
implementation right. Willison recently demonstrated
this by Vibe Coding with Swift UI.
Yeah, he gets this new M5 MacBook Pro, which he notes is
an absolute beast for running local LLMS because it has 128
gigs of unified memory. But he gets frustrated with the
built in Activity monitor on the Mac, and instead of just
complaining about it, he uses Clawed code to Vibe Code, a
custom performance monitoring tool from.
Scratch The significance of this project is Willison's
background. He comes from web development.
He is deeply embedded in Python And Django.
Swift UI is native Apple development.
It's a completely different paradigm with its own complex
state management and rendering. Rule.
Yeah, it's not an easy jump. No, but Claude acts as a
translation bridge. It maps Willison's known logic
patterns in Python over to the unfamiliar syntax of Swift
seamlessly. It fills in the framework
specific knowledge gaps so rapidly that a developer can
conceptually architect and deploy an app in a language they
don't even fluently speak. That is incredible.
And it's not just limited to user interfaces either.
Willison also built a functional Python vulnerability lookup tool
in just 40 minutes. He used the OSB dot dev open
source vulnerability database, which apparently has an open
cores Jason API. For my understanding, a course
Jason API allows a web browser to talk directly to another
server's database without needing a complex back end
intermediary. Yeah, and that is exactly why he
was able to build it so fast. Because of that API structure,
he didn't need to spin up a heavy back end service or a
database. He just fed the API
documentation directly to Claude and 40 minutes later he has a
lightweight front end tool where you can drop in a requirements
about TXT file or a GitHub repo URL and it instantly scans for
vulnerabilities. Building a functional security
adjacent tool in 40 minutes on a Tuesday is incredible.
But you know, the real test of that speed isn't building a
hobby tool, it's when you are actively under attack and every
single second counts. That brings us to Kellen
McMahon's live malware incident. Wilson highlighted McMahon's
story where he discovered a live malware attack happening on the
Light LM Pi Pi package. And Π Pi is the central
repository where all Python software is downloaded, right?
So a compromise there effects thousands of dependent projects
instantly. Instantly.
Yeah, McMahon actually used clawed in real time during the
incident response and he published the transcripts of the
session. He was feeding Claude the
heavily obfuscated malicious code to confirm the
vulnerability on the fly. He was asking it who to contact
and what immediate triage actions to take.
Claude was the one that parsed the situation and suggested the
specific Pi Pi emergency security contact address.
Having an AI as a real time incident response partner
fundamentally shifts how we handle security.
When you are dealing with a live malware attack, the cognitive
load on the engineer is just massive.
Having an agent sitting next to you that can instantly D
obfuscate code, parse server logs, and retrieve emergency
protocols provides an immense advantage.
It really does. You know, reading through all
these sources for this deep dive actually inspired me to start
building things this week. After reading about the Sentinel
cash bugs and the invisible API bills combined with Willis and
Swift UI post, I decided to try vibe coding myself.
Oh, really? What do you?
I'm currently using Clawed to build a little menu bar app in
Swift UI that hooks into my Anthropic account and tracks my
real time Clawed API spend. That way I have a visual monitor
right on my desktop to make sure the infinite intern isn't
bankrupting me. That is extremely practical.
Yeah, I'm doing something very similar based on Willison's
other post. Actually I'm building ACI
integrated Python vulnerability checker using that OSV dot dev
API he highlighted. Oh nice.
Yeah, I wanted to run automatically as a lightweight
security 30 gate before any code gets merged into my main branch.
It is just so empowering to realize we can read an article
about a new API on a Tuesday and have a working custom tool
deployed by Thursday. Yeah, it's a great time to be
building. OK, let's wrap this up with some
immediate takeaways. Listeners, we want you to be
able to protect your code and your wallets right now.
Based on the sources we've covered, here are three action
items you can do today. Number one, check your GitHub
Copilot data settings. Remember, the deadline is April
24th, 2026. It is opt out by default.
Go into your settings and make sure your proprietary code isn't
being scooped up for model training.
Number 2GO read that Reddit post on the clawed cash books.
Look at the workarounds they posted and seriously audit your
recent clawed API bills. Make sure you aren't paying 10
times the normal rate for an invisible caching error.
Critical and #3 Sit down and write or heavily update your
claw UEE dot MD file. Treat it like you are on
boarding a brilliant but completely silent engineer.
Put your bumpers up. I will leave you with one final
thought to Mull over an implication that wasn't directly
stated in any of our sources, but feels inevitable given the
trajectory we're on. OK, let's hear it.
If we are now writing our Cleelemd files specifically
formatted for an AI's comprehension rather than the
humans, and we are using AI to vibe code massive amounts of
boilerplate at lightspeed, are we approaching A tipping point
where our code bases are being optimized entirely for machine
readability? Oh wow.
And if that happens, what is going to happen when a human
inevitably needs to step in under extreme pressure and fix a
deeply flawed, sprawling system that was never actually meant
for human eyes? That is a terrifying thought.
From finding an advertisement hallucinated into a pull request
to realizing you might not even be able to read your own code
base in a few years, the Ferrari is definitely moving fast.
Just make sure you keep your hands on the wheel.
Thank you so much for joining us on this deep dive.
We love exploring these massive shifts with you, figuring out
the hidden traps and finding the incredible tools along the way.
Until next time, keep building, keep learning and please keep
checking those API bills.