This episode is brought to you by Forethought.
Most companies build their customer experience in pieces.
Sales in one system, support in another, and onboarding
somewhere else. Forethought brings it all
together. Forethought is an AI system
made-up of advanced agents that handle sales, onboarding,
support, and retention. Each team manages its own
agents. The customer sees one unified
experience. Forethought powers over a
billion interactions every month for brands like Scale AI,
Cohere, Air Table, and Upwork. Learn more at Forethought dot
AI. Hey, it's Eric newcomer.
Welcome to the Cerebral Valley podcast.
Our occasional detour from the newcomer podcast with me are are
now three time hosts Max Child and James Wilsterman, the Co
founders of Volley and the Co hosts of the Struble Valley AI
Summit. Welcome to the podcast, guys.
Thank you, glad to be here. Eric, happy to be back, excited
about the conference coming up in London.
Yeah. So we, you know, always sort of
jump into the Struble Valley podcast ahead of our Tribal
Valley AI summits. And we've got one June 25th in
London. People are like, oh, isn't it
hard to do an international conference?
And like, like everything in startups, it's easiest to do
hard things when you underestimate how difficult it
is until you're like, oh, yeah, we're just doing it.
Someone is figuring out taxes. We have an event team there.
I saw we have the ticket prices in pounds on the website so I.
Think exactly. Like someone was on top of that.
So a couple fun things for this episode for the long time
listeners. At the end of the podcast, we
will return to the startup draft, perhaps my favorite part
of the show. We have overtime accumulated
startups in our imaginary portfolio and get to score keep
how we're doing working backwards.
Before that, we're going to dig into some of our predictions
from the last series. This is sort of a mid year
snapshot, right guys, in terms of the predictions that we made?
Yeah. We made about 10 predictions
last year in November with the idea that they would come to
fruition within a year. So this is the mid year check
in, no official scorekeeping needed, but we will obviously be
competitive on our mid year check insurance as well.
In the latter half of the show, we'll be doing our, our games.
But I just wanted to start off, you know, we've been doing this.
We launched the first Cerebral Valley in March 2023.
ChatGPT had just come out and everyone was getting their heads
around models Max and James, if you want to sort of walk me
through, how do you think of sort of the thematic evolution
of AI in that period? And where?
Where are we now as we're programming for Cerebral Valley
London? Last time we chatted we it was
the end of last year, which feels like a long time ago in
the AI world. But we were all having this
discussion of if we were hitting a scaling wall and if the models
would stop getting better. And I would say it feels like
every two weeks since then, something amazing has happened
in AI. Is pretty trivial.
Has been a great leap forward. I actually think the theme of
hitting the scaling wall aged really poorly, which we somewhat
predicted at the time. But you know, maybe we didn't
know how wrong we were. I mean, just some highlights.
We had, you know, GBT 01, the sort of first thinking model.
We had O3, which has brought thinking models into the tool
world. We've had Gemini really take
great leaps forward and kind of become a state-of-the-art model
system. We just had Claude 4-3 weeks
ago, which a lot of people consider the greatest model
built so far, maybe the best coding model ever made.
We've had incredible Leafs in image generation from mid
journey, Gemini Flux, a bunch of other folks, and then of course
we're going to this episode 2. Video generation has also been
unbelievable. VO3 in particular, which is
Googles new video model I think is that the first sort of truly
realistic seeming video generation model in my opinion
and and kind of a great leap forward there.
You know, Alexander Wang at our conference in November was
probably the most prominent person arguing for a scaling
wall or a potential issue. I mean, he was talking his own
book. You know, he's in the post
training business. I do think there's an argument
that a lot of the progress, some of the progress, some of the
progress we've seen has been about post training.
The models are a certain level of smart, but then they sort of
talk to each other, they're corrected in certain ways and
that's where they get more intelligence.
That seems to be the improvement we're seeing with O3 and the
Chain of Thought models. Yeah.
I mean, I think, I think Alexander Wang, I give him a lot
of credit for what he was saying at our conference in November
because he was arguing that maybe we would see a scaling
wall to some degree in pre training, but we wouldn't see
performance levelling at all. And I think that's exactly
what's happened. And to your point, like a lot of
the gains from thinking models have come from new post training
techniques, reinforcement learning and new a whole new
scaling paradigm I guess. Yeah, I mean, in the end, the
models have gotten really, really effing good.
And whether it's pre training or post training, they go good.
Yeah. Yeah.
I feel like the vibe was a little more like maybe this is
the end of AI progress, not just like pre training is over.
And to your point, there was a distinction drawn by some of the
people at the conference, but I think there was a maybe a more
negative tenor going forward from a lot of folks on stage.
And instead, I think it's been maybe the craziest 6 months in
the history of AI. I don't know.
It's felt like that to me. The interesting thing is, I
guess like in January, we had that DeepSeek moment, right?
We haven't talked about that. I forgot.
About DeepSeek. Too.
Yeah, yeah. Wow.
So to some degree that was maybe echoing some of these scaling
wall concerns because if you can get such high performance out
of, you know, open source models that are effectively competing
with the frontier U.S. companies, like maybe there is
an argument there. But then I guess, you know,
that's kind of died down a bit as we've seen both anthropic and
open AI come out with superior models to some degree.
What models are you guys using? Applaud for Gemini 2/5 for
coding and then for my own personal sort of research and
note taking and stuff. Probably O3.
That's the same for me. I've been using a lot of deep
research through ChatGPT. I think that's my favorite
product maybe of the last few months.
I don't have time. I use O3 a lot.
I use deep research some. I don't want to blow anyone up
here, but I I had deep research, right?
A whole like political consulting memo for somebody I
was trying to get to run for office.
You know, it's like it's amazing.
I mean, there is a world where I would have gone and paid a
consultant to be like draw out for me, like when races will be
available and when, when they could run.
And it's like you just like, oh, in the morning you're having a
manic fit and you're like, oh, let's, let's see what Chachi BT
can do. It's it's insane.
It's crazy. This is a very nerdy, lame use
case, but I like to buy cheap wine that is still good.
And there was a secret wine from a local provider and they said
it's this anonymous secret wine that we're selling for 1/5 of
market price. But you know, if you bought it
at market price, it would be a 250, three, $100 wine.
And they were like, it comes from these amazing vineyards and
the West side of Napa and the foothills, blah, blah, blah,
blah, blah. And I just pasted the
description into deep research and I was like, figure out what
the secret wine is like 20 and 20 minutes later comes back.
It was like, obviously this is like this BV Latour 2022 cab or
whatever, and I was like, what? I've been super impressed with
ChatGPT multimodal for shopping. The three of us are all going to
the same wedding in France in a month or so, and I don't know if
you guys have looked at the required attire, but they're
half of. It Oh my God, my wife is very
concerned. Yeah, this hot tip is paste that
whole thing in a ChatGPT, ask it to shop for you.
It'll be great. My wife and I.
This was a good idea. I mean, it's not perfect, I
won't lie. Like how like maybe one out of
10 suggestions are like completely off base, but the
shopping integration, it just, it's kind of showing where
things are headed like, you know?
What do you mean integration? Well, because it's actually like
searching the web, you know, doing an agentic workflow of, of
looking for these items and then it's pulling that information
back into Chachi BT in line in the chat.
There's links that link out, right?
Eventually I'm sure you'll be able to just like add that to
your cart within Chachi BT It's a full shopping experience.
It's not just researching. Yeah, interesting.
He used the word. He used the word agentic.
Yeah. OK.
I mean, yeah, we're going to talk a lot, you know, over the
next couple episodes in terms of how AI can get put to use.
The topic I really wanted to get into this week before reviewing
our games and our scorekeeping is agents.
Like I feel like agents have been at once sort of the
buzziest thing in the backdrop of a couple events, but and I
never quite here. And so I guess the first direct
question I want to ask is, is deep research an agent?
Is O3 an agent? Like what, what?
What is an agent these days if it's just delivering you a
report? I buy the agent definition that
an agent is, you know, an AI tool that can actually do stuff
for you, right? That can go through some sort of
series of steps involving actions and you know, quote UN
quote tool use is sort of one of the popular phrases these days
of, you know, using different tools.
Currently the only tools these agents can use really are
essentially like web search and you know, maybe pulling shopping
links and showing pictures to James's discussion.
But I still fundamentally think like there is a big difference
between, you know, ChatGPT of 6 to 12 months ago where right,
you know, you ask it a question, it gives you an answer.
Essentially it's you know it's a text in text out engine, right?
You put text in one side, it gives you text out the other
side, but it doesn't go do stuff that's.
Part of that kind of web it's. Like, yeah, yeah, yeah, exactly
right. Even the deep research that we
were talking about, I mean, I used to be a management
consultant for two years, 2 horrible years, and it can do
much better versions of what I did as a management consultant,
you know, in a matter of minutes, it could do 5 days of
world class management consultant level research on a
topic, right? And I guess if you don't think
that's agentic, I think you're a little crazy, like you're too
high, right? It's clearly running around.
I I've seen some people use like time is it's like if it takes
time, how long it takes, you know, if it's all going and
doing things and interacting with the world and coming back,
I want to throw down a gauntlet to me.
And this could come soon. We'll we'll be in the world of
agents once people are letting them run wild with their own
credit cards once, once agents are spending money without a
human check in, that's when we've got sort of real agents.
What do you think? So does it.
Not count in your mind if like my agent finds me a dope pair of
shoes and, you know, text me, hey, can I buy this?
And I'm like, yeah, go for it. No, it needs to transact.
It needs. It needs to do without you.
Oh, truly? It'll be like this.
My great. Thing that's like that's an
agent. I mean, I understand like you
know, you, you book flights and an agent would have like asked
you before that. Definition I I feel like if it
does a restaurant reservation, maybe no money changes hands.
That's still an agent. I mean, I I just think this.
Is it's definitely an agent. I I'm just saying that's like a
great employee solves the problem, right?
It's not like, oh, they come back to you and want all this
feedback. It's like do the thing like pull
the trigger like once we can trust to do that.
I'm just saying that would be a landmark moment that I don't
think is so far away. Or do you think that is far
away? No, I, I honestly think that if,
if you had the capability to do that already in ChatGPT, like
James did his clothing research, probably some people would be
like, yeah, fine, go do it. Like don't buy too many things
without asking me. But like, I think people would
already be down. I honestly think the company's
reluctance is probably just mostly like they don't want it
to go off the rails and spend, you know, thousands of dollars
of people's money or whatever. But it's it's possible already
in my opinion. I don't know if it would work
yet. Like, and I think a lot of this
ties into context how much Chachi BT knows about me.
I mean it it's obviously, you know, starting to build that
memory, but I don't think it knows enough without me
prompting it or, you know, having very targeted list of
shopping ideas for this wedding to go just start buying me
stuff. I that'd be super interesting
once we get there, but I don't think it's ready for that yet.
Max just was sort of getting at this.
I mean, stop limits I think are key, right?
I mean, there's a degree to which algos, you know, I was
talking to a former banker about this the other day.
Like, you know, it's not crazy that we would let a machine, a
computer make payment decisions on its own.
Traders do it all the time. You just create some limits and
checks and hopefully have people hovering.
But like the algorithms have to move before a person could
react. And so that's happening there.
And so you can see with LLMS, it's like, OK, you build up a
trust up to the, you know, $500 limit.
And you're like, you can, yeah, it'll be interesting.
I mean, how many chargebacks are our chargebacks can go up with
everybody releasing power to agents?
And you're like, oh man, we need to do more refunds when the
agents do something crazy. I mean, I think that kind of
gets into some of this like quote UN quote agentic web
discussion, which is like, can we redesign the web in a way
that enables more of this behavior?
Because I do actually think if you're a clothing retailer,
there probably is some kind of business model in which you let
people buy way too many clothes and then they send most of them
back or they cancel most of them before they ship or whatever.
Right? Like, which obviously is along
the lines of like a Stitch Fix or Trunk Club or some of these
other companies which, you know, weren't wildly successful
because I think the return fees are pretty punitive or, you
know, and you end up with a lot of fraud and, you know, damaged
clothes and stuff like that. But I do think there's an
interesting question of like, could you build some sort of
consumer commerce website where people's agents can like buy way
too much stuff, like basically and then cancel it or return it
or limit it or some in some way because people buying too much
stuff by accident. And it's it's probably good if
you're selling things like even if you have to find a way to let
them cancel it or return it. My first reaction to what you
were saying is, oh, this is like a workaround to allow agents to
spend when we're really going to unwind it, but it makes the
default spending versus not. It's like, oh, the sugar is
good, which is which is really good for if.
You're selling stuff. Yeah, yeah, yeah, yeah.
James, what's the agent use case you're most excited about?
I can't ignore coding. We haven't talked about coding,
which I think is like the most actually valuable the.
Real one. That's why we want to talk about
it's boring. It's happening.
And that's. Something actually works?
Oh yeah, yeah, yeah, OK. AI podcasts are only about the
future, you know. Yeah, I think, well, I just
think that there's a lot to unpack about the future of the
of coding. I mean, I am a CTOI code when I
can and this agentic world has dramatically changed what I can
do in terms of prototyping and participating in the coding at
Volley and learning faster, right?
I mean, you just learn so much faster about different
technologies and tech stacks. And yeah, I think it's like if
you're not an engineer, you maybe understand this a little
bit or you've played around with things that are a little bit
more. No code like lovable.
Or figma or. Figma right?
I mean I. Think both Lovable and Figma are
speaking our event and they are now competitors in the the no
code world. Yeah, both non coders and
coders, if you've dabbled with any of this, you are, you know,
receiving the future of like what could happen to all types
of computer work, white collar work, if you want to call it,
but is happening first in in the engineering space and it's it's
pretty remarkable. I think Nat Friedman had a
really good analogy that I can't get out of my head about agent
tick coding. And he talks about the idea that
you have, you know, you have this room full of interns who
are all like junior engineers basically, right?
And you assign each of them a task and they go off and try to
do it. And then when they get stuck,
they raise their hand and they say, hey, I need help here,
come, come help me. I'm, I'm stuck with this bug or
or this this, you know, issue the, the app's not working,
whatever. And he podcasted, I think about
six months ago and he was like, right now basically these
interns raise their hands like every 5 minutes in like human
time. Like so they do like 5 minutes
of work and they raise their hand and you've got to go help
them. And then they raise their hand 5
minutes later and you know, over and over and over again, he's
like, so you can't really have met that many of these, you
know, imaginary interns going because you can just become, be
running around fixing their problems all the time.
You know, maybe only one really like because you're just
constantly having to give them feedback.
I do think like the frontier for like how much quote UN quote
human work these agentic coding tools can do now without you
having to run over and help them when they raise their hand is
like probably somewhere in the like 15 to 30 minute range now
where like, you know, they obviously come back to you very
quickly. Like they iterate through their
work very fast because they can type at a superhuman speed,
right? They can put out hundreds of
lines of code. But the sort of amount of human
level work I would say in my testing that they get stuck is
probably like, you know, some of that 15 to 30 minute range.
I'm confused, like are you saying 15 to 30 minutes of like
what it would take for a human intern?
What it would take like a good human engineer like, you know,
Yeah, I got it. You know, mid to senior software
engineer just just hammering away at code, right?
Yeah. Like how much code do you get
out between them getting stuck? Right.
Like, yeah. I would say yeah, maybe half an
hour of like human, human work, but.
You get it. You get it in 3 minutes or two
minutes you get. It in a minute, you know, or 30
seconds usually, which which is which is mind boggling.
But like the dream is you get, you know, 4 hours of human human
work or 8 hours of human work or eventually, you know, weeks or
just starts fixing itself and you never go in there, right?
I mean, you know, I spent a fair bit of time vibe coding a month
or so ago. I do think there are some self
driving car aspects in the sense that like the last 5% or 10% of
a problem is very important. And like it's like, oh, it looks
close. It looks close, but it's like,
sure, it's really good when I'm like copying the sub stack
design, dropping a lovable like, oh, rebuild that.
But like at some point I feel like you've just been working
long enough and you have some minor tweak you want to make and
it just starts getting stuck and has no idea.
I mean, partially that's why I'm trying to do no code and no code
knowledge. But I do think these programs
are really good at being enticing in the beginning, and
then they sort of get overwhelmed as the project
starts to expand. Maybe this project that you were
doing, you won't be able to ship because of that last five mile
problem or whatever. The fact that you even started
it is kind of amazing that you did that.
Like you wouldn't have been able to even start the project three
months ago or six months ago or something.
Sure. Yeah.
So I don't know, it's just kind of interesting, like where that
heads is. You know, I think eventually you
get to the point where it's a finished product and then
there's just like way more coding projects happening in the
world, right? And then, you know, I think
within companies that already have an engineering workforce
like it actually can get across that that last mile.
I think another important theme that always comes up is like if
we froze this moment in time, how much value is there that
people are still understanding versus depending on continued
progress in the models that's that's revolutionary.
Where are you guys on how much harvesting could be done with
what would already exist versus what we're waiting on?
I would say a lot like a lot of harvested progress.
Like I can't even think like trillions.
I don't know. I haven't decided of 10s of
trillions, but let's say trillions.
What's the measures? In value.
Oh, like GDP. GDP value of just harvesting the
stuff that we've done so far. But it's amazing you still see
people who are like, skeptic. I mean, I don't know.
I just don't want to be, you know, deluding myself here.
Yeah. I mean, I think like
fundamentally we've all had the experience, right?
Where something that used to take 8 hours now takes less than
a minute, right? I mean, like we've all had that
experience, right? And so if you just do the math
on that, you say, OK, I took something that used to be a 500
minute problem and now it's a one minute problem.
Unless you believe that thing had no value whatsoever, how
could you not see a 500X in knowledge work in front of you
and say, hey, maybe we haven't harvested all the value on this
500X improvement, though this thing I just did.
How could you not believe that that's going to have insane
ramifications throughout our world and and insane amounts of
value that eventually can be created.
You know, so I don't know. I just to me, I just look at
that, you know, 2 orders of magnitude differential we're
already seeing on labor and think it's got to be huge.
Here's a question for you guys. Like, I have a friend who's in
PE and he, you know, doesn't really, he's not really in the
tech world that much, but he, you know, tried chat GPD didn't
really, you know, find it super valuable.
He had never tried O3. And I just showed him what was
possible and he, like, you know, kind of changed his whole
opinion of what types of companies he should buy in the
future. Wow.
Well, I do think that's a real danger that there are, there are
things that I'm changing about my life and medical decisions
and lots of stuff off O3 BS. It's so persuasive that we'll
never really be able to back out the psychological.
And, you know, it's all right. It's like, is it having an
effect? It's like, yeah, it's deeply
affecting big decisions in my life.
You know, just because it's like the thought partner, it's there
just like, you know, if you were to have the friend that's there
while you're soundboarding an idea, like that's going to be
dramatic. And I think just the fact that
it's in the loop. Wouldn't you say overall that
it's in person giving good advice?
That's why I keep it in the loop.
Yeah, right. It's not like, but but it will
be hard to know if it if it hallucinated.
I don't know. It's like, is it gonna?
Yeah, hopefully in a couple of years I'll chase after me in
this shot. GPT will be.
I told you a couple of key things that I think you took to
heart and and now that I'm a little smarter, I'm gonna go
back on that. Yeah, we'll see.
Having lived through the iPhone experience of of having the
first iPhone in 2007 and having to spend the next four years
explaining to my friends why they had to get an iPhone and
why they. Should probably.
Consider getting an iPhone. I mean James, you had a
BlackBerry until what, 2011 or something like that?
Like 10/20/11 like. James, I'm sorry, I was the
lager to. I had a BlackBerry for a long
time. Yeah, and people would be like,
well, Max, what? Why?
Why do you, I need an iPhone. What do you do with an iPhone?
And I'm like, well, you can like browse the Internet.
And they're like, well, you know, I browse the Internet on
my phone. It's fine.
And I'd be like, well, these apps are pretty cool.
And they'd be like what? Like the the app that makes it
look like you're drinking a beer and I'm.
Like, well, I'm not. Using that every day, but just
the last point I have to make on this BlackBerry thing.
I'm sorry, BlackBerry, which obviously is a dead company
today. No one you know has a
BlackBerry, right? Their sales continue to grow for
four years after the iPhone was released right?
So O 708-0910 eleven BlackBerry sales continue to increase even
though we all today look back and even in the movie about
BlackBerry, they pretended that the day the iPhone came out it
was like P BlackBerry. Sorry guys, you missed the
future. And I'm just pointing out that
in the real world, sales continue to grow for the next 4
years. And so I'm just saying that when
you're in one of these sort of Roadrunner, Wiley Coyote chases
the Roadrunner over the ledge moments where you take a second
before you look down and realize that gravity is pulling you
there. I think similarly with AI, we're
all like, holy crap, we just went off the ledge.
Like shit is about to get really dramatically different here.
And you can still stand there in the air for a year, two years or
three years or four years before gravity really hits, right.
And and I think to your initial question, like we are in that
period where even if nothing changed, like we're already off
the ledge, you know, stuff, stuff is going to be
dramatically different. No matter what happens from
here, which I know we all believe, you know, the models
are going to continue to get better.
So the pace of acceleration is going to be even higher.
It's a perfect endnote. Let's move the conversation to
our predictions from six months ago.
Basically, we're not going to get 2 in the weeds.
It was a fun discussion. James, do you want to take us
through them, What the question was, where we each landed and
then we'll give a quick reaction and then go to the next one.
Sure. Sounds good.
Just to clarify also we back in November, we asked Claude and
Chachi BT to generate these predictions for us including
providing probability estimates of how likely they were.
And then so our job was to take the over or the under on each
prediction. The first one was open AI shifts
GPT 5 with a greater than 10 trillion parameter model Max was
the over, Eric under and I took the over.
Thoughts. Where is it?
GPT 5 here. Yeah, so far.
So far I'm correct. Six months.
I mean, I think we much hung up on they wouldn't use the name or
maybe there was a high chance that they would abandoned the
name. It seems like they're going to
do it honestly right now. What the vibes are that they're
going to kill the O series and just make 5 the overall?
I feel super good about the name and it's shipping this year.
At this point. I think the only thing that
could hit the under would be the 10 trillion parameters because
didn't Brad, like at the CEO literally say they were going to
ship GPT 5 this year and it was going to be called GPT 5?
Like I'm pretty sure he said that like last.
Seems like a. So I would still smash the over
the 10 trillion parameter. Thing is, I guess the one.
Certainly I would buy the over now.
I might take the under because of that 10 trillion like I oh.
You're I'm sticking to my bet. Whatever I we're the best locked
in. Yeah, I can't be wrong.
I was trying to figure out like so Claude 4 Opus just launched
and there's no comment on the parameters that I can find, but
the best estimate I could get from Claude was 2 to 3 trillion,
so I don't know if that's nice. Nice.
I mean a lot of the improvements seem to be post training and
chain. Of thought and if they're.
Going to merge, you know, the O series with GPD 5.
It's partially because they think they need part of the
improvement off these reasoning models, and so maybe it's a sign
that the parameters aren't getting as big as we thought.
Also also it seems like adding that many parameters you know
creates huge issues with inference cost and just serving
them it's. Expensive.
Yeah. Anyway, the next one was three
or more countries enact national rules regarding AI medical
diagnosis. Max, you took the under, Eric
took the over, and I took the over.
We had it at a 70% probability. You guys are crazy.
There's no way you think in the
next 6 months three countries are gonna enact regulations.
That happened so far, medical diagnosis.
Here's what I had. Oh, you.
Found something chachi BT is saying.
Yes, EU AI Act. Exactly, Yeah.
UKSMHRFUS to regs online chachi BT things we're like in good
shape all. Right.
Well you took 70% so never forget that I get 2 to one.
Yeah. I don't know how you're going to
do the overall math at the end, but I'm sure whatever.
Onward and upward. All right, Tesla full staff
driving approved for unsupervised driving in one or
more US state, 40% probability. Max had the over, Eric had the
over, and I had the over. I think we're in good shape
there, right? Isn't Texas gonna happen like
yeah in June, supposedly. Like, yeah, in two weeks,
correct? But it does seem like right?
Chachibi says not yet Pilot cars on private roads in Texas.
No public permit. It's coming to Austin in June.
Now. I think this gets into the
letter of the prediction, though.
Is it a state law or city law, whatever.
OK, we'll come back to that in a few months.
AI will write the copy for greater than 50% of a major news
outlets articles. 30% chance Max had the over, Eric had the over
and I took the under. I just feel like my, my under is
great here. I mean, nobody's.
Nobody's. Claiming this nobody's first
newsroom. Everybody is like, what's the?
Eric's gonna do this by the end of the year.
Just to go build. A new media company.
I won't be major though. I don't think I'll I'll qualify.
You could be major by the end of the year.
Why not? Apparently, the Chicago Sun
Times inadvertently ran an AI generated book list filled with
errors, sparking backlash. Clearly not paying for 03 there
I think. I just think we're gonna see so
many of these things, like hallucinations, like, aren't we
already seeing this from the Trump administration?
Like just random things that don't make any sense.
I certainly don't want, you know, AI to replace the
newsrooms. I'm just expecting a lot of like
stories over the next year about like academic papers and news
articles just having obvious hallucinations, right?
Because people are going to be using.
But shouldn't we be more disturbed that the Trump
administration is like, I feel like if Democrats were in charge
and they were releasing government reports that were
clearly written by AI, yeah, it would be like the biggest
cultural story of the moment. It's like, well, the governments
already phoning it in with Trump, it's like, oh, at least
they're using it is. Literally what I was going to
say. I was like, I would honestly
prefer the AI to be doing this than the miscellaneous Trump
administration employee. Like knows nothing about
anything. I just want Sam Altman to give
every administration staffer free O3 so that they have O3
right. Reasonably intelligent fake
reports instead of real ones. I do think this core take that
the Internet split everybody's view and this is part of why
originally Marc Andreessen was so pro crypto and anti AI.
I do think AI is going to bring us potentially closer together
where people are asking rock like is this bullshit true or
false? I mean, it's possible people
build crazier models, but for now, while the models are sort
of generally in agreement with each other about how the world
works, it could be a major force for cultural consensus over the
next couple decades. It's basically the network TV of
the Internet essentially, right? We're all watching the three
major channels and they all broadcast kind of like exactly,
but not family friendly content. And your take is that Marc
Andreessen? Marc Andreessen doesn't want to
bring us together. He was weird, crazy shit.
Yeah, that's what crypto is all about.
Like, do whatever you want, like, and AI is a source of
conformity. But then Andreessen Horowitz
basically saw what was winning, and it was like, we not need to
go where the money is. And, you know, that's how I see
the story playing out. I mean, they were resistant at
first. Khosla led the open AI venture
round. Anyway, keep going.
Didn't Peter Thiel say like AI was communist or something?
Yeah. Yeah, communist and crypto was
libertarian or whatever. Right.
OK, Fully AI scripted and AI rendered feature film gets a
theatrical release 25% chance. Max with the under, Eric with
the over and James with the under.
This was this was Eric using his insider knowledge of pay to play
tactics within the movie industry to to try to grab us on
this that someones going to pay a theater chain to take their
movie even if it sucks. Basically, make sense?
I mean, we still have time. I had to double check this that
it wasn't hallucination, but according to chat GVT there is a
fully AI generated movie releasing soon, Pirate Queen
Zheng Yi Sao, billed as the world's first fully AI generated
feature film. Exactly.
That's going to get a festival run.
I gotta watch it. All right, moving on, more than
three major smartphone OEMs chip phones with AI Co processors
running 7 billion parameter plus models on device. 7 billion is
pretty high. The rumor mill on Apple is
saying they're going to be able to run 3 to 4 billion.
So even if you believe in James's claim that the chip
they've had in there for 12 years is an AI chip, it still
might not be able to do a 7 billion parameter model.
So we were specifying an Apple, Samsung, Google, Chami,
dedicated AI Co processors that run 7 billion perimeter LLMS
fully locally. We had the probability at 75% to
unders Max and Eric and I took the over and I yeah, I was
counting the the current technology as capable of running
those types of models. But yeah, to your point, Max, I
think 7 billion might be a bit high, right?
3 to 4 is what the rumor mills saying for this year, but we'll
know in a few more months I guess.
Next. Anthropic releases a model
scoring over 90% on U Bar, the unified benchmark for AI
reasoning, which does not exist according to our own research.
During the podcast recording last year, Complete Hallucinated
prediction from Claude. How are you feeling about that
one guys? Great.
I should, yeah. Certainly raises red flags,
yeah. Moving on to #8 / 4 Fortune 500
firms 5 or more cut greater than 25% of their middle management
roles by the end of this year, crediting AI explicitly with a
25% probability. Actually interesting.
Here we have an over and over from Max and over from Eric and
on under from myself. I'm feeling pretty good about
the under. Yeah, yeah, look, I'm.
Gonna do the research on this. We're only asking for five firms
to cut 25% of only middle management.
So that's a pretty that's a low bar.
According to my research with ChatGPT, this has not
materialized. Many large companies are
experimenting with AI and none have reported cutting 1/4 of
their management. Yeah, maybe we overestimated,
first of all how much we thought it would give them air cover for
all sorts of things. But yes, exactly.
Maybe we didn't want to get right into the AI narrative.
If we got a tariff induced recession, this actually might
happen so. OK #9 Deepmine and Google
discover a new drug that clears phase one trials within 2025.
We gave that a 20% probability and Max took the under, Eric
took the under, and I took the under.
I think all looking pretty strong here.
And D mine spun out isomorphic, which would be the start up.
I think that would potentially do this.
So there's a there's a chance that even if it happens, we can
all claim technicality that it doesn't.
But I I think it's, it's not looking likely, right?
They have to be in phase one trials already.
Yeah, I mean I. Think we?
Yeah, I think that they are planning to enter trials by the
end of this year, so unlikely to have completed phase one trial.
And lastly, we have the international AI treaty with
greater than or equal to 15 signatories, including three of
the US, China, EU and UK. 50% probability we all took the
under. Seems like a good bet so far.
Yeah, the AI believes too much in human institutions.
Right, because US, China both seem unlikely, right?
Is Europe getting EUUK viable getting the US which under Trump
is now like no AI regulation and China which is we do what we
want. The fact that the EU and UK get
separate credit here doing a lot of work, but three of the four
seems high do. You know, do you know what the
score is? So just pulling together the
scores, I asked Chachi PT to create a scoring system.
It came out with a formula inspired by Breyer style scoring
which I had never heard of but. Yeah, that's how close you are
to the probability. Yeah exactly.
Yeah, seems like a good scoring system.
It gives 3rd place to Max Dinged for betting against AI film
better calibrated on his conservative bets like hardware
and policy. 2nd place Eric solid instinct, slight overconfidence
on a few misses and in first myself.
Great balance of bold but accurate calls.
Thank you, Chachi I. Love it, love it.
I think I think this AI film take is complete bullshit.
So I need to I need to score adjusted for that.
This is just a check in, no medals awarded yet, but I will
take the pole position and see you guys in a few months where
we can do the final tally. All right, let's do our fantasy
draft. Max, you want to talk us through
the game and then we'll get into our picks.
Yes, we invented an ingenious game based on fantasy football
that allowed us to draft teams of startups into imaginary
rosters. We've done two different drafts.
We did one about a year and a half ago and one about six
months ago. We restricted the draft list,
the draft board as it were, to companies that had raised over
$100 million at the time. So if there's obvious omissions
that come up in your mind, it's probably because they hadn't
raised 100 million at the. Cursor Cursor.
Being the most. I think like there was some like
where we've that we didn't include because.
Yeah, We also excluded specific like chip based companies,
Chinese companies, I don't know for robotics, I can't remember
healthcare, we anything we thought we were even Dumber than
normal about we left off the list.
So we all drafted teams. We did a snake draft.
Most notably, the first draft involved a discount that we had
one person had to take for getting the first pick because
the first pick was very obvious. It was Open AI.
Eric paid, I believe, $75 billion in handicap to draft.
Opening auction 1st and I and I really good about it.
Aged pretty well since they're valued at 300 million right now
3. 100 billion, would you say? 300 billion.
I'm sorry. Yeah, like coughed.
Yeah, it doesn't matter. And I will say before we say our
teams, I have not yet had the first pick in any draft.
So I just want everyone to remember that when making our
teams, OK, I will go through my team first, which I will admit
upfront is in last place right now.
All right, my team, Databricks, my star worth $62 billion,
Cohere AI, model company, modular AI, language company
Scale AI, Sierra AI, Sakana AI and Hebia.
And if you know all of those names, you are far too online.
I'm bullish on Sierra. I think that's, I think scale.
Has room to run as well. Oh yeah, yeah, and obviously
Databricks isn't going anywhere. But Anthropic surpassing data
bricks has been a bit of a sob story for my team because James
got Anthropic at Crazy crazy money if I recall.
The 4th pick that wasn't even your third pick because it's a
snake draft. So it was me then Max the data
bricks, then James with you'll say in a second, which doesn't
make any sense, and then fourth with Anthropic.
It's so embarrassing in retrospect, just.
To just to clarify, like we drafted these teams originally
in 2023 and then we did, I don't know, we drafted it.
We, we had an ad drop waiver period last November and this
again is a mid year check in. No, no ads, no drops, but
checking in on the teams. So Max, I have your score so far
right now at 93 billion, mostly because some of your teams have
not raised or exited since you drafted them.
I have no valuation on Sierra and scale is at 25.
I'm somewhat optimistic on both of those.
I managed to somehow pick the only foundation model company in
the world that isn't wildly overvalued.
Cohere SSI thinking machines, Anthropic.
Like just throw a dart boarded foundation models.
You've got a $40 billion company.
But I'm. So Silicon Valley, the show said
this from the beginning. No.
Revenue is so much better than revenue.
Cohere is a real business, so people can value.
And. SSI and thinking machines are a
dream. I have made a huge real.
Revenue I I think SSI has real revenue, but anyway.
Yeah, Max, what's your learning from this so far?
I would say we already knew this, but you know the winners
keep winning, right? Obviously Open AI swamps
everything else that's happened in the entire draft.
So we have a true power law which is nothing else matters
comparison to Open AI even with the handicap which.
Which we knew, which we knew was the risk when we came into it.
Was happening, but regardless it still happened.
Secondly, I would say Databricks is, you know, a merely a $60
billion company, but that looks lame compared to, you know, like
XAI being valued 80 like. You know, all right, all right,
let's not spoil. James, you want.
To go next. My team with the first pick on
my draft that you guys were making fun of just the moments
ago hugging face, No value because they have not raised
since 2023. I've drafted them because they
had one of the highest valuations at the time of the
draft. They were valued, I think, over
a billion dollars. I thought they were valued at 4
or 4 billion. Dollars at the time, yeah,
something like that, yeah. Yeah.
OK So Anthropic giving me 61 1/2 billion value replit hasn't
raised since the original draft. I exited Adept at 1 billion.
I snagged XAI with the first pick last November.
I'm. So jealous of that.
Locked in 80 billion of value right there because they raised
earlier this year and runway also raised $4 billion
valuation. 11 Labs also raised at a $3.3 billion valuation and
poolside no raise recently. Also one of those foundation
model companies that we have yet to really see appear on the
draft board, but I am happy with my overall team and my score of
close to $150 billion currently. You know, something I just
thought about, You're extremely lucky that XAI purchased X and
not the opposite way, because if it had been X, it purchased XAI.
We'd be able to like, force you to disown whatever growth, but
now you get to benefit from this combined monstrosity, which
kudos to you. I would just say had I been able
to draft first, I would be in James's position of being in
second place with XAI, so I don't personally think that a
coin flip should be dictating my performance right now.
What is you? I know.
What was me I I will say to give James credit here, I I truly
believe that anthropic is is the pick of the draft or, you know,
so far I think that just getting anthropic at the fourth position
in retrospect looks insane. And so I think that that is the
the greatest. I don't even know if I call it a
sleeper is sort of a semi sleeper pick, but that that
clearly to me has had the most appreciation.
Eric, why don't you all? Right.
So yeah, I picked Open AI with a $75 billion handicap.
Now it's worth 300 billion. So I'm getting basically 225
billion for that. Inflection sold for 1.43
billion. Character sold to Google for 2.5
billion. Glean we're scoring at 4.6
billion, but rumored to be raising at 7 billion.
Miss Straw AI worried about that one. 6 billion right now.
Perplexity. Oh man, I'm getting no credit
for that. That's going to be a good 10
right now, but it will be 14 is apparent according to the rumor,
so we'll see. Safe super intelligence.
I knew this was buzzy, but I don't even know if I could have
seen this one raised at $32 billion already.
It's more perplexity like That's insane.
Kodium sold for 3 billion do. You want to explain that it's
they renamed to Windsor for their name.
Kodium is windsurf. Yeah, they sold to Open AI and
then Harvey. No credit right now but rumored
to be raising at 5 billion. Total value $274.5 billion.
Yeah. I feel really good about this.
I mean, hysterically, as I think I mentioned on the last episode,
I I wrote a bear case about open AI after this at 157 billion, I
think, but whatever. So I'm getting it narratively
both ways. But yeah, I mean, I'm proud of
all my picks. I think even my sort of singles
are selling and I'm bullish on basically everything except I
would like to hear what's going on with Mistral.
But yeah, it's I mean, it's a power law business.
It's crazy that like I'm like, oh, glean that's that's a good
company. I was totally right that that
would be a good company, but it it doesn't really matter for my
my performance. Yeah, I mean I think you are
consistently hitting singles and double s, but you could have
nothing on your team except opening eye and be beating us by
100 plus billion dollars at this point.
So it. Doesn't, no.
Which is why I made a that we could not randomly assign the
first one and I I said we know. You were right.
So I'm complaining, I I'm not complaining.
You made the right decision 100%.
It's just it, it is remarkable. It's like the whole game is just
like, open. AI who drafted Open AI who
drafted one Open AI go back. To that episode.
Yeah, exactly. Yeah, No, we talked about it.
I mean, I think it was it would they were raising at 90 at the
time. And so you ended up with a $75
billion handicap on a $90 billion company, which seemed
like a reasonable deal to us. But you know, we were.
We were all wrong, obviously. Or at least we're wrong with us.
Check in. These do have to last, right?
It's like 5 years or something. We're we're like, yeah.
Five years, yeah. Yeah, we had a good shot with
Sam Hoffman getting fired of your team.
Going up sync. But but then he came back in
force. All right.
What's come on the market that you think we'll be looking at at
the end of this year? Thinking machines.
Cursor for sure. Manus.
Oh yeah, yeah, yeah. Thinking Machines, cursor and
manus are the ones that come to mind for me.
Yeah, I mean, it's, it's going to be a tight band because we're
we're picking them up and we're only interested in ones What
that. They have to have raised $100
million. All right, well, that's that's
basically our episode. We're gonna have two more before
the Cerebral Valley AI Summit in London on June 25th.
And then at some point, once we've gathered ourselves, gone
to that wedding in France we mentioned and relaxed a little,
we'll come back to you and give you our thoughts from the event.
I'm super excited about next week.
No pressure James, our our game master over here, but we're
trying to come up with some good concepts, but we'll be talking
about voice and video and certainly in light of what VO3
Googles new video creation model, it's an exciting time in
video. So see you next week.
Thanks guys. See ya.
Thank you.