Welcome back to In the Loop and after what feels like years of
waiting, GPT 5 is finally here. So about a week on opinion is
quite divided. You've got many people saying
this is revolutionary, but just as many people saying it's
disappointing. You had on the launch event open
AI showing bar charts that were just straight out lies.
And you have prediction markets betting against open AI just
minutes after their launch. So if you're sat here thinking,
I have no idea what's going on, well, that's probably the only
reasonable response you could have.
In today's episode, you're going to learn everything that is
worth knowing about this new model release, what's new,
what's improved and what was just a lot of good marketing.
We've spent many hours collecting all the research and
reviews so you don't have to. This is in the leap of Jack
Horton. I hope you enjoyed the show.
So before I get into the episode, I think it's important
to set the scene and add just a little bit of context to this
model release. For the past year, we've been
living in basically model chaos. It feels like Open AI has been
throwing spaghetti at the wall when it comes to language
models. We've had GPT 44.5, the O series
with different specialised models, and much of it felt
sometimes like they were trying to justify a $200 per month
price tag for their pro membership.
And people were often so confused about which model they
were going to use for this specific task of theirs that
they had to even release white papers on it.
And meanwhile, Anthropic has been completely eating away at
their enterprise market. So particularly in coding, we've
covered this in a number of episodes, but called code and
Anthropic Clod language model has become the go to choice for
developers across the world. And I guess the general take
away has been quite clear, which is most people would go to
ChatGPT for general use. But for most people, if you're
going to go do a coding task, you're going to go and use clod.
And in this context frames the release of GPT 5 really well.
This release wasn't just a better model, it was all about
cleaning up this chaotic mess of different language models and
trying to take back market share that they've lost over the last,
I'd say 69 and 12 months to clock.
Now with that being said, let's get into it.
And I think it's going to be really helpful to start with a
high level overview of some of Chachi BT5's new features and
abilities. So one of the first things
you're going to notice since using the new Chachi BT5 model
is that it's called a hybrid model.
And the reason this is noticeable is because it's got
both thinking and non thinking versions built into the exact
same model. So previously if you asked a
question, unless you specified the O series or the thinking
model, you wouldn't have any reasoning and and thinking in
any of the responses that you get from it.
You'd have to specifically choose a reasoning model.
And for those on a free tier, you've only got a limited number
of queries on that. And in fact, initially, they
actually called to deprecate many of their former models, so
they completely took away Chachi, BT 44.1, the entire O
series of models. There's been such a big uprising
about this that Sam Altman's ended up releasing what is
basically a small essay of tweets and open AI, actually
bringing back many of the models that people loved into the
subscription package. For those paying $200 a month,
this could be an episode on its own just due to the amount of, I
guess, emotional attachment that people have clearly had to some
of these models and the fact that it was then taken away from
them caused real upset online. Another new addition is that
you're going to now be able to short circuit it's thinking when
it goes into a thinking loop for a question that to be honest,
was quite simple. It's also going to be available
to everybody, so including all free users.
I think they really learned from the Deep Sea Saga last year that
everyone must experience their best models.
And there are also three versions of Chat Chi BT5.
We have the standard one, the Mini 1 and the Nano.
So if you then hit usage limits on one model, it will
automatically switch to another model.
Also, it's a massive context window, which is for me a huge
improvement. It's context window is now
400,000 tokens, which is larger than all previous models.
And if I put that into context, that's about 300,000 words.
And again, if you listen to the episode on context engineering a
few weeks ago, context is the ability for the agent to have a
set of information when you ask it a question.
So it could be a long part of a copy and paste of information.
In document terms, that's about 600 to 800 pages of text it can
keep in its short term memory when you're having a
conversation with it. And previous models were about
128,000 tokens, so that's 96,000 words.
So that jumped from 96,000 words of information it can store in
its memory when you're having a conversation with it to now.
300,000 is a huge, huge leap. There's nothing more than more
annoying than putting information into a model and
hitting a wall because of a context of it.
Another huge addition has been coding and front end creation
just using ChatGPT. They've clearly gone all out to
also try and take Liverpool's market share.
It now has a really good understanding of spacing and
typography and even white space and design.
You can vibe code entire user interface really, really simply.
And in fact, in the actual press release, which I really highly
recommend you go and check out, they provide a load of prompts
that can generate really cool applications.
And this is the exact reason that Sam Altman recently came
out and said that this is the fast fashion era of technology.
We'll get into this a little bit more throughout the episode, but
you want to keep it top of mind. They've also really lowered the
barrier of price. It's now much, much cheaper.
You're paying about $10 per 1,000,000 tokens.
And output tokens, by the way, is I guess is the number of
words it uses when it responds to you.
For the same 1,000,000 tokens that you pay $10.00 for with
Open AI, you're actually having to pay $75.00 for Opus 1.
So that's Claude's good model. So that pricing is absolutely
crazy. It's 10 times cheaper than
Claude, which is, to be honest, really difficult to say no to.
And it's actually been reported that API usage for companies
using ChatGPT inside their application or existing
workflows and systems has doubled in the last few days of
the ChatGPT launch and kept on growing subsequently after that
launch. It's been reported that the
usage of Open AISAPI, so that's using the language models inside
existing software actually doubled within just the first 48
hours of ChatGPT 5 coming out. This is clearly an aggressive
pricing strategy designed to win back market share, especially in
that coding space where Anthropic has grown rapidly.
And finally, they've launched a what they described as a unified
system. But that basically means that
they've added a router, which quickly decides which model is
going to be used to perform that task.
And that's going to be based on the complexity of the tool
types. So if it needs to do a web
search tool or maybe the type of question that's been asked, all
of that sounds really cool. And to be honest, the main area
they've invested in clearly is the coding space.
And to be honest, if you just watched their entire launch
video, if you had no context on what Open AI did, you would
probably assume that they are predominantly A coding
application. So as we go into what the
independent benchmarks say, I think that's a really important
thing to keep top of mind. Now, as you will know, I'm not a
massive fan of these benchmarks, to be honest.
I think it's a game that all of them play and they're all
obviously going to try and make each other look as best they
possibly can on launch day. But I think it helps gives us, I
guess, an initial gauge of whether there's been a big leap
or not. Artificial analysis, which is
the gold standard for independent AI benchmarks, found
that when ChatGPT was tested and it used its reasoning, it was
the best model out there. However, when GPT 5 didn't have
reasoning, it performs closer to GPT 4.1's level.
This part is because actually what it shows us is again, the
open air isn't selling US1 model.
They're selling an entire new packaging and pricing system.
This minimal version that most people, to be honest, will often
get is basically just their old models, and their high reasoning
version that gets the incredible flashy headlines about coming
first place in the most important benchmarks is going to
be rarely used. So through very clever pricing
and packaging, they're trying to increase the perception of value
across all their tiers. And if you think about what that
means for say, competition, Anthropic might charge premium
prices for Clawed because they position it as consistently high
quality. But Open AI has built a new
packaging and pricing system that enables them to undercut
Clawed and other model providers, all by using that
router system, which is automatically routing people to
different models. And subsequently, Open AI has
announced that they're going to bring out the ability for you to
understand which models being used more often.
Which for me is a really big win because I personally haven't
been trusting the quality of responses.
And I can't help but think, am I getting the rubbish models and
still paying money for it? When it comes to coding
benchmarks, Vellum's analysis found that Grok 4, GPT 5, and
Cloud Opus are all performing a very similar level on the SWE
bench, though ChatGPT did come first place in many of the other
coding benchmarks like Design Arena.
And so even though ChatGPT has led to the number one best
performing model in many coding benchmarks, Group 4, So Elon
Musk's language model still actually gets the best marks on
some of the hardest types of assessments on the ARC AGI two
test, which is designed basically to measure abstract
reasoning. That's more like human problem
solving, the kind of figure it out yourself thinking that
humans have to do when they've never actually seen a specific
problem before. Well, Grokforce scores highest
on that assessment at 15.9%, while ChatGPT 5 actually lags
behind it, less than 10%, so 9.9%, which is a huge gap.
So in short, ChatGPT 5 is faster, it's better at coding,
it's cheaper, but across many of the reasoning benchmarks, it's
not as good. That's just the benchmarks.
And as we all know with language models, it's just as much about
the vibes. And certainly there's been a lot
of noise and conversation about those vibes with ChatGPT 5, and
that's included very divided opinions.
So on Polymarket, which is a decentralised prediction market.
So you can bet on whether Apple's going to bring out a
foldable phone in 2026. They actually placed 80% odds on
Open AI having the best leading AI model by the end of August
before the demo. And then Google was lagging
behind it, just 20%. But by the time Sam Altman and
the Open AI team ward off that stage, the title completely
turned and Google surged to 85% and Open AI to 14%.
So that only 14% of betters believe that Open AI will have
the best model in 2026 from 80%, 14% and this speaks volume on
the divided opinion that people have got around this new model
release During the ChatGPT 5 release, there was also let's
call it chart crime Open AI showed off several charts on the
screen as they spoke to demonstrate how good ChatGPT 5
really is and it honestly left me baffled.
So it's hard to explain, but if you want a podcast platform that
has video will actually show the bar chart because it's it's
hilarious. They had bars that were next to
each other but drastically different percentage outcomes.
So we had one bar that got a 69% score on coding deception,
another one next to it that got 30.8, but they were displayed as
identical bar heights, while the Chachi BT5 results were 52.8%.
Yeah, was visually showed as being much, much bigger than the
other bars that were at 69% and 30%.
It's criminal. Someone addressed it afterwards
and said it was a mega chart screw up from them, but I guess
many people were claiming that they vibe coded it using
ChatGPT. You have Professor Ethan Mollick
predicting that you're going to see a lot of varied results
posted online about chat GB 25 because it's using multiple
models in its responses, some of which are very good and some of
which are, to be honest, meh. And since the underlying model
selection isn't very transparent, it becomes very
confusing. And this is exactly my
experience so far. They also had to do a massive
patch just after their release because usage went through the
roof and everything started to fail, including that auto
Switcher. So switching to different models
and as a result, it just started putting every single person on
the worst possible model. This blog by Matt Schumer, I
think, captures the broader market reaction really, really
well. And he said, when I started
using it, I wasn't blown away. And in fact, I felt very let
down given all of this hype and a massive expectation around it.
But what Schumer said was that if you're using Chachi BT5 for
tasks that the previous O3 models or even the Cloud Sonic
can do really well, then you're not really going to get a gauge
of how good it is to see how good this truly is.
You have to ask it to do things that AI simply can't do very
easily. And in short, that is coding.
And I think this gets the meat of what open AI I've clearly
obsessed over getting right, because the vibe coding
revolution is here, and that means it's the era of creating
things all of the time. Before I go into some of the
examples of what's been built on ChatGPT 5, I just want to
quickly stop and acknowledge that we're growing incredibly
fast. And according to the stats, only
45% of you listening right now are following us or are
subscribed. So if you could press that
button on your podcast app, that would help so, so much.
A shed load of effort goes into every episode and that number is
part of what makes it worth it. So the Vive coding revolution
really is here. And Matthew Burnham, who has
480,000 subscribers on YouTube, actually focuses all on AI
coding and he got access to the model quite early and he built a
bunch of really, really cool things.
So you had ChatGPT 5 generate several very specific
application to test its vibe coding abilities.
Again, if you're watching the video, we'll put them on the
screen. But he made a jumping ball
runner game. So a simple game that although
is very basic, was functional. He created a pixel art tool, so
an entire application that you can draw things on and it can
change colours. He said it was a bit laggy, but
not too bad. There was also a game he
generated which measures your typing speed, so which is a
functional typing test that essentially tracked the accuracy
and speed of you typing out words.
The game even included proper feedback and scoring systems.
He also created a drum simulator, which is an
interactive music application that you could literally make
music on and finally a tool that visualises music and lo fi,
which is also just really, really cool.
So the fact that you don't need to go to another application
now, ChatGPT literally does this for you from a single prompt.
And this is the world of vibe coding.
Every application is now a vibe coding application.
I think another fantastic review and one that I wanted to know
because I really, really enjoyed the reading was won by the
Leighton Space and it was one of the more nuanced and
philosophical pieces on ChatGPT 5 and they called it the Stone
Age. This wasn't a criticism.
This is a reflection of where they think we are in human
civilisation right now, because this is one of the first model
that excels at using tools during conversations, just as
humans did a long, long time ago.
So as they put it, the Stone Age marked the dawn of human
intelligence, but what exactly made it so significant is the
fact that we shaped tools and our tools shaped us.
So I guess what does all this mean for Anthropic?
So the threat to Anthropic is really interesting because
apparently according to reports both Cursor and GitHub copilot
drive approximately $1.2 billion of Anthropics 4 billion revenue.
So that is just two applications built and using clawed every
single day as part of their application that is driving most
of their revenue while 30% of it anyway.
And with Chat GPT's new model coming out and it's incredible
abilities with coding and costing 7 times less than Claws
models, well, it's going to be hard for an enterprise customer
to say no. And lots of coding applications
like Livable Cursor and others have made ChatGPT 5 their
default model again because Claude has brought out Claude
Code, which is actually a competitor to many of the
companies that are using their language model.
Many of those companies are now jumped straight to Open AI to
make it their default model. This doesn't mean that
developers will not choose Claude still, but it's certainly
a massive risk to Anthropic. So I guess to conclude, ChatGPT
represents both impressive technical achievements and
really importantly, very aggressive competitive
strategies against Anthropic. For coding, it appears to have
made a huge leap forward, but for other applications the
improvements feel incredibly incremental and to be honest,
overall disappointing. This was just as much of A
business model change disguised as technical leaps because Open
AI have just built a system where they can offer cheaper
language models to everybody with minimal reasoning, while
still managed to claim that they have the very best model on the
market. And obviously it's working.
Cursor switched overnight, prediction markets lost
confidence in Open AI and Anthropic is watching 30% of
their revenue become potentially vulnerable to their pricing.
And I guess anecdotally, the chart crimes, the very mixed
reviews, the expectation gaps are probably a symptom of now a
maturing market where these revolutionary AI narratives just
don't quite work in the same way as they used to.
The real story here is where the ChatGPT can write better code,
write better content, and solve real problems for everyday
people. Anyway, that's it for today.
I hope you enjoyed that deep dive into ChatGPT 5.
Thanks for listening and I will see you next week.