Welcome back to Cerebral Valley. I'm here with Max Child and
James Woolsterman, my friends and Co founders of Volley.
This episode we are talking about chips and big tech.
My interview, after a conversation with Max and James
is with Chris Miller, the author of Chip War, The fight for the
World's most critical technology.
He is an expert. We talk a lot about NVIDIA.
Yeah, so stick around for that. Max James welcome back.
Nice to be here. Eric, Glad to be here.
The starting point question that I wanted to frame things up with
is, do you think GPU capacity, the quality of these graphics
processing units and what NVIDIA is putting out is the main
driver for the sort of generative AI revolution we're
seeing right now? What do you mostly attribute it
to? The research papers, The new
approaches. How?
How do you? How do you?
Who do you give? Credit.
I guess I give more credit to the research papers.
Not that I don't discount what NVIDIA has done here.
I just, I think they were in the right place at the right time
with the right technology to accelerate what was possible
through due to the research. But yeah, the research was the
breakthrough. I do think there was some
intentionality. I mean, it's always hard to tell
with AI when there's so much. It's a hypey thing to talk
about. And this is ACEO that also made
a bunch of money off crypto mining.
So happy to dive into wild speculation.
Clearly I don't know. Max, you have a take here.
I mean I think I agree with James.
I just think that, you know even four or five years ago we were
already starting to see the beginnings of this with GPT 2
and you know earlier versions of the the foundational models,
right. Also in areas outside of large
language models you saw big progress in, you know our hobby
horses of speech synthesis, speech recognition and other
areas. So I think that even with five
year old GP US, you still could pretty to do pretty cool stuff
with the new research that was outright whereas.
If the research never comes out, I don't think that you're making
this kind of progress in, in any of these kind of large large
language model areas that we or or any of these areas of the
stack that we've seen so far. So right.
I mean, chips, even though there's potentially exponential
improvement, it's still like, you know, on a trajectory,
whereas a paper sort of comes out of nowhere, gives people a
new approach and then sort of revolutionizes what's possible.
I mean, I think a different way to think about it would be like,
OK, GPT has gotten like 1000 times better in the last three
years and like the chips haven't gotten 1000 times better in the
last three years. So like, what's driving that?
Like, it's probably the software and then the people like
figuring out how to make GPT, right?
So yeah, I just think it's clearly based on the ideas in
the software. The chips are great, no doubt,
but it's not the key driver. I I do think it so NVIDIA has
clearly gotten rich off this stuff.
It is now a $1.1 trillion company.
In one year the stock is up 258% and in five years it's up 564%.
So this, this has been a great run for the company.
I mean what do you, what do you guys make of NVIDIA investing in
all these companies? I mean similarly if it's into,
you know Amazon, Google, like all these companies are making
investments. The money that they're spending
obviously comes back to their business, right.
NVIDIA wants AI to be super active, so puts out money that
gets spent on NVIDIA chips. Yeah.
What do you make of, I mean it's round, it's round tripping in
the OR it's you know spending money to make money, but it's it
doesn't seem as sort of negative as in some cases.
I mean, isn't the investment NVIDIA is putting out into the
ecosystem like a rounding error compared to their revenue or
compared to their profits? I mean, like, I I mean I think
that they're making such ludicrous quantities of money
that kicking back like 1 to 2% into the startup community that
might invest in NVIDIA chips in the future is, is obviously just
a very good business practice, right.
Right. Well, it seems like they're
literally playing kingmaker, right.
I mean like companies like Core Weave and who are sort of chip
doing chip adjacent stuff depend on their access to H1 Hundreds
and a lot of these foundation models seem like they're
fundraising on the premise that hey we have access like this is
differentiated partially because we have the deal to get GPU
access when other people don't. And so there's this hope that
you know, you're you have a Moat in that you have access to to
chips that other people don't. The scarcity today of the best
in class chip, the H 100 is. Has to be a temporary situation.
I just like the economics of like making more of these chips
is a very good idea for NVIDIA. So they're going to do it.
But they they clearly want to build like a Amazon Web Services
type competitor. And so having eyes and ears in a
bunch of startups and seeing maybe who they could acquire to
do that is smart. I do think the interesting
question again is like Intel basically had like 2 decades of
like total monopolistic dominance of of PC chips, right?
And. The reason basically was that
it's like really, really, really hard to build fabrication plants
to to build CP US. And it took a long time for like
TSMC and other people to sort of catch up on that front and then
eventually kind of surpass them in many ways.
But the question I guess for GP US I think is like, is it that
technically hard to make GP US that like whatever the next
version of the H 100 is? You know that they're going to
be on to that before anyone's even caught up to the H 100, and
so they'll have this insurmountable 2 two to
three-year advantage where if you want the best AIGPUS, it's
always going to be NVIDIA for the next 20 years.
Because that seems like a plausible case to me.
Do you think Microsoft was smart to invest 10 billion in open AI
given it's clearly going to distract the company as has been
reported? I think my friend in the Wall
Street Journal reported as much that they directed sort of their
attention to open AI away from Microsoft.
And and part of what you're saying is they they are giving
up access to their own GPUs to Open AI.
Is that accurate to to say so? Yeah, I mean, I think it was.
I think it was a good decision and.
I agree. Definitely a good decision.
I I can't imagine they would be, as you know critical to where we
are in the in the AI era right now had they not made that
decision they. Have you know early access to
all the technology that Open AI is creating.
They got GPT 4 into Bing before it was in public in you know in
front of customers for in front of Open AIS customers and seems
to be continuing with what they are trying to do with Office and
Windows, you know the Office suite.
So I think. Just that early access to Open
AI products seems worth the investment.
I think, I think the sort of meta question you're asking that
I think is really interesting is like.
Is big tech going to win the AI race or whatever, right?
Like is like, I mean we always used to talk about Fang right?
Like Facebook, Amazon, Apple used to be Netflix and Google
right? You could replace the N and Fang
with NVIDIA and it would be perfect.
So Fang OK and like, goodbye streaming.
Hello. Chips.
Yeah, Goodbye streaming. Hello.
Chips. Right.
And I love Netflix, So they never got over like a couple 100
billion, which is where you said, you know, NVIDIA is a
trillion now. So pathetic. 200 Three $100
billion company. Yeah.
Is fan going to win AI? Like, I think that's a really
interesting question. And I think like I'm kind of
leaning, yes, I guess to your point, like it seems like
Microsoft's been able to grab, you know, Open AI, which is, per
our last podcast, still with the most valuable asset in the
entire industry, right. It seems like Amazon's getting
very cozy with Anthropic, in case that's actually the other
big asset in the foundation model category.
Facebook is absolutely like ripping on open source models.
You know, Llama. They're going to put like LLMS
inside their ad products so you can have automated ads on
Facebook and Instagram and all that stuff, you know?
They're also putting, they're also putting LLMS into consumer
chat. You know, products, right?
You can chat with celebrities, and I'm sure they're doing a lot
with image generation. Yeah, exactly.
And then Google seems to be like fully ripping on, just like
dumping large language models into every single Google
product. Like Google.
You know, Google Docs, Google Photos, I don't know if that is
the conventional wisdom. And so on.
OK. Google ripping for Google.
For the record, you have to like grade them on the Google curve,
which is they're in a competent company coming out with new
ideas. Given that Google has not come
out with a new idea in 15 years, they are doing a tremendous job
putting AI products into their existing, into their existing
products, right? And so, yeah, for them, the
number of launches this year has been tremendous, right, Graded
on the Google curve, right. And then Apple has basically
done nothing. But Apple always does things
like 2 years late and then hopefully they get it right.
So the, the great dream of Apple as I understand it is just that,
you know, we get to a point where a lot of these small
models can be run local, local and that they have these great
M1 and M2 whatever generation we're on now chips.
And that even though they're not GP, they're sort of combined
chips, right. So there is some graphical
component, right. And so that those chips would be
sort of pretty capable at handling sort of small local
models? Totally.
And I mean. Apple's argument would be we're
using transformer based models in photo recognition stuff,
we're using it in speech recognition stuff, we're using
it in auto correct. They just put it in the iPhone,
right. They're like their their take is
always like well we will do it when there's a real customer
need and this tool can really provide value.
So I think like Jury is still out there.
I also think with Apple Vision Pro it'll be really interesting
to see what kind of generative AI and and also just you know
the AI driven tool set that launches with Apple Vision Pro.
So I think like. It's a really interesting
question as to whether or not like this is whole AI revolution
is just going to entrench the power of the big 5 tech
companies because I think like they look pretty good right now.
You didn't. You didn't mention Amazon,
right? I I said Amazon was like cozying
up to anthropic. And also, I mean, you can
elaborate on the other stuff they're doing, which is quite
copious. On on Alexa specifically.
Well, and also AWS, right? Bedrock and everything, right?
Yeah, right. Well, it feels like Amazon, I
mean Microsoft is clearly ahead like they with Azure and AWS for
those two companies I think are the most important.
I get you guys really care about Alexa and stuff could happen.
But for the businesses today, like Amazon Web Services is so
essential. And like Microsoft can go to
people and say like Azure has a direct relationship with Open
AI, whereas Amazon, it feels like they've been pretty slow on
this. But like like you said, they
have Bedrock and they're trying to come up with these sort of
partnerships, but. Yeah.
Well, their strategy is they're going to put everyone else's
model on AWS, right. So they're going to be the
middleman, like they're going to sell you other access to other
people's models. So the anthropic models and I
think to the Llama models and to whoever other models they can
get on there, they're going to get everything on there.
And the only like maybe they will, maybe they won't end up
with open AIS models on there. I think that's an interesting
ongoing business discussion. But like, in the end, they're
going to try to be. You know the supermarket for,
for, for models in the cloud, right as they have been in the
the the last generation of all the different web tools they
sell. So like.
But maybe maybe Azure is not allowing Open AI to put that
model on AWS essentially right? Yeah.
So is Open AI so good that that will cripple Amazon?
But it is interesting that I imagine most developers that who
are using GPT models are. Doing that directly through the
open AIAPIS. Yeah, probably.
And there is, you know, there's this world now that was already
trending this way of big companies wanting to have like a
foot in Azure and AWS so that you can sort of get the best of
both worlds. But that would seem to be good
for Microsoft given that they were not the first place player.
So even saying OK this further creates this world where you
want hybrid cloud or you want to dual app clouds is is sort of
good for Microsoft. I I can't believe I'm talking
cloud computing on the podcast. I feel like even even even as
long as we're willing to go it's funny to be in that room.
OK. Facebook, I think part, you know
Facebook is releasing Llama which is sort of their, they're
sort of the king public company and open source models.
Their their their top AI person is Jan Lacun who's sort of he's
been at this forever and has had a lot of spicy and open takes.
I mean I guess the the core question is just like what do
you make of Facebook which is sort of a closed social network
being so pro like let's just give, let's give foundation
models away, make it easy, let's commoditize them.
Let's just like give away. What do you make of the Llama
strategy? Like what's?
What's the point of giving away Foundation models and making
that such a core piece of your approach?
I mean, I love it. I think it's good.
Yeah, I think it's great. I think it's really, it's really
probably primarily about developer relations and creating
goodwill in the open source and the developer communities.
That for them is more important overall to the like longevity of
their business than. Owning you know, or monetizing
these APIs, these models through APIs which would be a just an
entirely new business for them. So I don't know, that's that's
my take. I don't know.
Do you guys have a different perspective?
I just think that I think that there's basically 2 strategies
here with like open and closed source software, right?
One is like closed source where you keep it private.
You're like we have the best stuff.
And people will have to come to us even though they won't know
the sort of how the internal workings of the software are
because it's the best, right. And that's pretty much the open
the eye strategy, right, like and some of the other foundation
model companies like Anthropic whatever.
And then Facebook is like, OK, somebody already took the closed
source. We're the best strategy.
We're going to take the the, you know, opposite strategy which
is. We're going to open source all
these models, right? And we're going to hopefully
build developer communities around these models so that
people can make them better. They can, you know, fine tune
them. They can like, you know, help
optimize them over time. And then we will have like the,
you know, the. Thousands or, you know, 10s of
thousands of developers who are working on top of these models,
like build and improve them and build a network of of plugins
and, you know, optimize them and all this good stuff.
And so we're going to get value out of the developer community
and and therefore the models that our business Facebook is
built on top of will will get better and better that way,
right. And so I think it's like it may
not work, but I think it's better than just trying to.
It's differentiated. Yeah, It's differentiated,
right. Yeah.
And you know, there are lots of great startups that have been
built. You know where it's like we're
going to give away a foundation, we're going to give away, sorry,
an open source project and then you could build a business
around if it's super successful. Also, for Facebook, they
probably benefit a lot from hiring great AI talent in the
long run. And and if.
If that alone just gets them, you know marginally, you know
percentage. I know Reels is getting better.
You know, I I feel like the. The reels is good.
Reels is a good algorithm. When if Facebook and Google nail
AI generated ad advertisements, right, or add just even the
words in the advertisements, the copyright like that, you have to
imagine that's a huge tailwind for the business, right?
Like like. You know, personalized ads that
are created by AI in real time as you scroll to Instagram
feeder or the Facebook feed or whatever.
Like, you have to imagine that's.
Yeah, yeah, you have to imagine that's a crazy good business for
them, right? Or improvement to their business
and lowers the friction to actually buying the ads.
It's kind of interesting if you think of it that way, because
what you're saying Max is AI just improving and getting
better faster. Is, is is what's best for
Facebook overall, almost like if we had a breakthrough in, you
know, energy production of energy or clean energy or
something that would be great for Facebook too, probably
because they would, you know, have a lot, we could run their
data centers a lot cheaper and you know all these things,
right? So it's almost just like better
for them to encourage the advancement of this industry
faster cause a lot of the gains will accrue to them.
We we jumped into this with business strategy which I am
enjoying. But to me what's interesting
about Facebook's positioning is that you know Google, Microsoft
with Open AI to some degree they're playing the responsible
steward right. It's we are going to make sure
our language model behaves appropriately, doesn't get off
the rails. We're our brands are at stake
and we need to really guard them.
You know, Mustafa Suleiman at Inflection, who's going to be
speaking at the conference has sort of staked out and you know,
we're going to interrogate this more.
And so I don't want to caricature it before he speaks.
But like a somewhat open source, skeptical position, right?
It it undermines the idea that we need to really be thoughtful
and protective about who gets access to what and what these
systems can do. And so Facebook is playing sort
of AI, don't know flamethrower fire thrower type role here
where they're throwing fuel to the fire on open source in an
area where a lot of the other big tech companies would say the
thing we can do is like do this really well and then keep it
really closed off so that, you know, we have basically we're
almost as functioning as a state ourselves where we're really
being thoughtful. What do What do you take of
Facebook, the big company playing this role of
democratizing? I do worry, I guess for them if.
Someone uses Llama for nefarious purposes like.
Essentially, it hasn't. It's a brand, but.
Yeah, I think that could happen. I think that this could be like
a long running issue now for them going forward the next few
years. If for if, if, as we've talked
about in previous episodes, these models can be used for
nefarious, really is laying the groundwork for reporters here.
You're like, OK, people should be held accountable for what
their foundation models, their open source project does.
I don't. I don't.
Know if I I don't know if I personally agree that I'm saying
that will be the. That could be a narrative for
sure, and it just depends on how how much harm can be done with
open source models, and I think we don't know yet.
I mean. Facebook's like, look, our brand
for trust and safety is already so bad that there's nothing we
can do that could possibly make it worse.
So why don't we just go for it On the business front, they're
like Cambridge Analytica. Wait till they see what people
will do with this model. Well, Cambridge Analytica is a
great similar analogy because they thought they were.
At the time they were opening up the Thorngren graph and they
were, you know, giving data to, you know, other developers and
companies. And they were, and that was, you
know, encouraged almost at the time, right, by developers.
And it blew back on them entirely.
And it wasn't even good for their business to be doing that,
to be giving, leaking that data. So it is there is kind of a
similar thing occurring here potentially.
Facebook's willing to be wild. Well the narrative kind of can
change on you right that with at the beginning of social
networking and and technology this was encouraged that we
would want you know more more data sharing or something with
developers and that and then Fast forward 5-10 years and it's
not have we all staked out points of view.
I think we're all on the same page on do we believe in open
source like data bricks, super pro open source, like a lot of
the, you know, obviously hugging face clam at CV1, super pro,
open source. Like, I feel like generally
we've heard from a lot of pro open source people.
Do either of you have reservations?
Are you fully in the camp of like it's an arms race and just
like get the best stuff out and let good people sort of take
advantage of? It my take is that open source
right now is great for the community, for developers, it's
awesome to be able to. You know, edit weights or use
smaller models, run, run in or your own inference, all these
things fine tune, right? And that just might not be the
case going forward. I don't know, at some point
maybe I will change my mind and be like, no, the models are too
powerful. Like I don't want these open
source models in the hands of nefarious actors or you know,
China or something. So I I guess my take is it's
great right now. I don't know, but it's hard to
pull. You know, that's sort of the
once they they have the models and it's gone too far, it seems
hard, hard to unwind the clock or whatever.
Yeah, I guess I I just believe the nefarious actors in this
scenario are unstoppable. And as we discussed in The AI
Kills Us All episode that somebody's going to figure out
how to you know, some bad, bad actor is going to get control of
a a really powerful model regardless of whether or not it
was open source or not so. Yeah, I mean give the, so you're
you're very pro give the pro open source give the give the
fruits of the labor away to the community.
I mean why do open source? Why do we think?
Why do we think Open AI changed its tune so dramatically on
this? Question.
Awesome business strategy. Just like, just protect your
business. Good for the front runner,
right? Like like like regulate it.
Like let tell. Make sure you have to spend a
lot of money, like following rules like that's good for big
companies. Because once they realized how
good the stuff they were cooking was, why give it away?
I mean, open source is the strategy when you're not the
leader. Like not, you know, you never,
never give away the the high quality stuff, yeah.
Or or the strategy that makes sense for meta because they have
this these huge other businesses right like you would.
It doesn't really make sense. To give away models as a start
up I guess I I mean maybe maybe that's what stability is kind of
doing or mid journey but you said they're kind of like
becoming more close source. Imagine a world where like these
algorithms are near AGI like and then they are locked down.
You know that's the ultimate elite.
So like a a small group of companies gets to control like
this sort of Infinity resource like it seems that seems insane.
You know what I mean? I feel like that would become
sort of one of the great freedom questions of our time, that just
in any way restricting the set of people who have limited to
access to like a God like reasoning and information
system. I feel like we're we're
backtracking on Oh yeah kills us all episode but but I mean I do
think open source was key to Facebook and that's how we got
down this path. But reasonable that we've gone
far from that though you've been the one that makes the point
that like often these discussions undercount the AGI
potential and how disruptive that would be to any of the
other arguments that we're. Having I total, I totally agree.
And that's where I would question your assumption that
everyone should have AGI. Like Without knowing how
dangerous that could be for the world, that seems like pretty
presumptuous to just assume that that it should not be regulated
by a handful of companies. Yeah, I mean it's a lot just.
Give away the God like powers. Everyone deserves God like
powers. I mean, everyone deserves what
can go around. Yeah, I mean they they
presumably they will have some counter, you know
countervailing. Presumably, sure, yeah.
I guess if that's the case, you know, but, oh, I guess here's an
easy way to make sure we haven't missed anybody.
Let's Let's do a collective ranking of who we think benefits
the most from AI in terms of market cap movements experienced
and to be experienced. This is I'm putting, I'm putting
in no we're doing it together. I'm putting.
I think NVIDIA should be our number one.
I mean, Microsoft would be the close one there, I think.
I mean, Microsoft stock isn't actually up anywhere nearly.
So, so NVIDIA then probably, I mean to your point I guess, but
I guess the question. What about Google?
I I would put Google pretty high here, Microsoft.
Depends on what time frame. 34% over the year and 187% / 5
years? Google you're putting Google.
I don't know. What to me?
What time rules in though like tech becoming more important and
Google being all over it is good for tech is good for Google.
But like Google's relative dominance, which was so powerful
in search, feels threatened by this, so I I just.
No, I think that's worth talking about because I just feel pretty
confident that that whole search impact of AI has like is like
very, very early and Google will have plenty of time to
potentially continue dominance with you know using LMS and and
AI and multimodal models in in in their search products, so.
They have, they have the the front page right that everyone
goes to for search and they have really powerful AI internally.
It just hasn't played out yet. But I don't.
I believe that they will get there.
All right. So I mean you're you want Google
first it just. Depends what we're talking
about. Like are we talking about which
of the biggest companies will have the highest market caps
from AII mean or just like relatively like you know?
Is their position improved by AI?
How much the percentage of their position is improved by AI?
That's what you're asking. Yeah, I mean NVIDIA.
Yeah, NVIDIA for sure. Microsoft second agree.
Facebook Third I would go what? Yeah, I don't.
I don't agree, because algorithms are core to what they
do in their discovery. Mechanism.
I think if you actually, if you oh, because of algorithms are
core to what they do, OK. I just think if you believe
there can be a 50% improvement in advertising at scale through
AI generation, then Facebook benefits more from that than
anyone except Google and and they might think more than
Google. I have just put Google before.
OK, I mean, that's fine. It's, but it's a totally.
Agree with Eric, but but. It's just about search versus
the Instagram feed. Which one gets more benefits out
of AI like, I think that's the real question here, like or
search versus the Instagram feed plus the Facebook feed, right?
And I guess it also has YouTube which is competing in like
shorts. Yeah.
But it's going to take a while to do video generation, I guess.
I don't know, like it's sort of about the latter of how quickly
generated personalized ads kind of get scaled up.
So it's. Yeah.
Facebook, you're mostly talking about their ad products and.
Ranking algorithms. But there's also an argument
around on the content generation piece itself that as we've
talked about before, there's, you know, ability to create more
compelling content and social media posts and I.
I really think the only way you can make the case for Google is
that you would say the ads on Google are going to get much
better, right? I mean like.
Well, what about search itself? You think search is going to get
so much better I. Think search itself is going to
get so dramatically better? Yes.
Like I think, I don't know, I mean time spent in search you
would think that go up if they're chat type products.
Well, well, there is an argument that time spent goes down,
right? Because you're just getting your
answer faster at an end. But it's you're not.
It's no longer a link. It's like less of a link system
and more like in the search, which is where Google ads are.
Well, it just depends. If it's more, it becomes more
like a ChatGPT model where you're chatting with Bard or
instead of searching. I guess that's where I'm coming
from. I I see me and myself doing
that. Like almost more than.
I searched Google today in the future, and I think they're the
best positioned to be that like true assistant that I use every
day to talk to and instead of searching by typing into AI mean
I was down on Google before. But like Google versus Facebook,
I'm getting heated up just in the sense that, like, if you
think of every product that Google offers like if Gmail was
amazing from AI and like the prompts were great and there was
writing like could become an even more insane product.
Obviously Like Google cloud services.
Wait, sorry, it's Google. Cloud platform, global cloud
platform is key. If they can improve their like
search, YouTube, like they're just so many pieces of their
business where they have audience and they could put this
to work. The only question we all have
probably is execution on all of these areas, but the opportunity
to me seems massive. I mean, I guess I'm just going
based on the fact that Google makes no money on anything
except search and the other stuff is great, but it's really
just a data vacuum for for search ads, like they really
don't make any money. I mean, they make one on YouTube
now, so that would be the other big thing.
But like search ads is the whole ball game if you're just purely
talking about the business of Google, pretty much.
And so you have to believe that search ads are going to get way
better and or that people are going to spend way more time in
search, which again is an execution question about like
Bard versus all the other, you know, large language models and
chat models. So like I buy the case, I just
don't think it's as clear cut as you guys do because I just think
Facebook like Instagram ads and Facebook ads getting better
would like to immediately dump right into the business.
Which and what you're saying is that there's more of an obvious
trajectory for. The Instagram ads and feed to
get better from AI than there is for search ads to get better
from AI. Yeah, yeah.
I mean, it's close. They're, you know, they both are
going to get a lot better. So, Yep.
But it's about which one's bigger and what's the bigger
impact, I guess, Yeah, notably, notably like, so yeah, we, we,
it's Facebook or I don't know, I Facebook wouldn't necessarily be
my next. Either.
Oh, really? Interesting.
I'm. I'm willing to.
I mean, I. Just feel I would bet it forth
at least, but what? What's next?
We have Apple and Amazon, you know, like are now towards the
bottom of our at least tech list.
Obviously tech being better in some ways lifts all boats, but I
don't know what do you make of those two and like what it means
for them. I would put Amazon next.
I think they have an obvious role to play with AWS and they
have made a lot of great partnerships there with
Anthropic already. They are probably working
internally on models themselves. They are the number one cloud
platform. They will benefit in their other
core businesses through AI potentially, you know Amazon.com
and ads there, which is a increasing part of their
business and not to ignore it, you're allowed to talk about it.
Yeah, Alexa. Which is critical to our
business. They have an end point in 30% of
US homes that is an AI assistant.
It historically hasn't been amazing at talking to you
beyond, you know, controlling smart home features and and
telling you the weather and playing volley games.
But it's it's completely possible that they will be able
to. AD LLMS and and AI experiences
as they've already demoed that would transform that device in a
way that would be you know essential would would would be a
game changer for the Alexa business.
So I think they're the furthest ahead in that area on like
potentially integrating AI with, you know, your average day
through a Voice Assistant. And Apple, I mean Apple, I mean
the argument Apple gets away with everything in that we can
put them in like basically last place and it's still like, well,
they move slowly and maybe they'll figure something out.
And like, I don't know, it's definitely their wheelhouse.
It's like it's not, it's a lot of data.
It's not about privacy. It's sort of not a device.
I mean, I guess, well, I mean, if we really want to criticize
Apple, we saw this report that, you know, maybe Sam Altman and
Johnny I've and Masa son Masayoshi, son of SoftBank are
getting the team together weird. The weirdest team headline you
could imagine to build a phone or some device that's sort of
got AI at its core, that that's a threat.
I mean, I could buy, you know, a next generation sort of AI first
device. I I I just think Apple's Moat
has always been how unbelievably difficult it is to build
hardware at scale, right? Especially like high margin
hardware. I mean, there's almost no other
large hardware businesses in the industry.
And even if Johnny is on board and Masa is on board, you still
need like Masa's got a great history.
You still need, yeah, you still need about three to five years
and 30 or $5 billion to build even like a small scale, high
quality hardware device. And just like I think that's
just a long way away and anywhere where you know Johnny
and and. Sam are playing like Tim Cook
will pay some attention to that and make sure that they're not
completely caught off guard, right.
You know, they all talk to the same people manufacturing the
stuff in China, so but would you?
Agree though that like Apple, while they're not threatened
that much, like relatively, they are probably lowly ranked here.
Yeah, I think. I I think.
They're the most, most protected on the downside in some case, in
some cases you could argue and then also they don't have huge
outside this is all bullshit. They're fine.
And if it's good, you know, it doesn't feel like they're going
to disrupt it, yeah. Right.
I would like to throw another company in the mix here.
What do you guys think about Tesla?
Ah, yeah. Self driving.
The old self driving and the general xai suite.
Now, you know, it's like he's somehow allowed to like, merge
AI research, like it seems like across all his companies.
I mean, I just think that I think I'd be much longer way MO
and cruise than than Tesla or whatever.
I mean like, I just think that. Tesla's fundamentally built
around a hardware business model where you sell cars and like if
you believe we're entering a world where all every car is
self driving and it can take you anywhere.
Like why wouldn't you be long? Like, yeah, Uber or Lyft or
Waymo or Cruise or somebody who's built around this model of
like providing cars as a service rather than cars as like a
jewelry object that sits in your garage 99% of the time.
Like, I just don't like. I think Tesla's a great product,
but is it an AI for self driving first product?
I always thought. The idea that it was gonna turn
into a self driving taxi fleet was like pretty absurd, but.
I think that's a good point. I think Elon, his hype or
marketing spin on this is that the AI that they've built to.
Successfully get self driving. Working at Tesla is so valuable
in future endeavors, right? Like robots, robotics.
But didn't they literally admit they gave up on all the AI
they've been building for? Until a year or two ago, they
had been using the last generation of deep learning
tools or whatever. And then at two years ago, there
was a. Like recently there was a story
that came out that basically said, yeah we had to redo the
whole thing once we realized like transformer models, large
language models are better, right.
So like they they've been saying that for like 7 years that like
we're way ahead and everyone on building a self driving models,
we have gazillion trillion miles of data blah, blah, blah.
Which is fine is true. But like they had to throw it
all in the trash can like 18 months ago.
So like I I don't find that like a super compelling like case
that they have some Moat around that sort of stuff.
But you know, who knows? I mean, they do have.
They don't have a lot of trust with like regulators.
Like, I don't. I feel like, you know, Waymo and
Cruz have just tried to be as responsible as possible.
You know, I mean, Waymo has been very conservative in deployment.
And you know, Cruz is like General Motors, like a company
that I feel like every American politician is rooting for,
right. I mean, Tesla, it seems like now
has been able to position itself as very close to the Republican
Party, but it feels like the administrative state and
Democrats are going to be, like, extremely skeptical of them just
throwing cars on the road. I mean, that's sort of cynical
even about, I guess, the politicians I like.
But I yeah, I think I agree with Max's point of technology.
I also think having a reputation for being a responsible, sane
actor is important when you're doing something as dangerous as
deploying the first self driving cars.
I think it's a fair, fair assessment.
I mean we'll see if they pivot the attack, they're going to be
a stock premium for it. I'm sure like I guess they pivot
the you couldn't you can't I don't know how you can ever
measure this because it's like well the stock can stay inflated
for like forever So if they if they pivot.
The entire company to robo taxis and they start selling $5 Tesla
rides all around America. I will totally change my tune on
this, but you pretty much have to embrace a disruptive business
model in the next three to five years, which is historically
been very, very difficult for any type of company because
you're kind of saying all this money we're making a day.
That's going to be 0 in five years and we're just going to
have to jump on the new thing and ride it out, which is really
tough. I to be clear, I don't think the
self driving deployment cycle will be anywhere near as fast as
that would suggest. So I don't think it's going to
destroy their like existing car business.
But I agree with the idea it's hard.
The the classic start up situation which you're
articulating is that like people don't disrupt their own good
businesses and so Tesla's in sort of the worst situation for
that. Just a counterpoint to that, do
we believe that other companies are going to be able to sell
fully self driving vehicles anytime soon?
It doesn't doesn't seem like it, at least besides the three we've
talked about. No, I I mean those other
companies seem to be only focused on the robo taxi market
as opposed to getting consumer vehicles that can fully self
drive themselves into the world like if.
Tesla is the only game in town there, and I'm not sure if
that's the case, but right now it seems to be the only one that
is going in that direction. That could end up being a very,
very compelling offering if the only car you can buy that will
drive itself anywhere is a test big if we'll see.
I feel like Waymo and crews are going to deploy city by city
geofence where they know they can do it with no deaths, you
know? Yeah.
Anyway. Yeah, are there any other, I
mean this is great. I I'm glad you brought up Tesla.
Like what Any more edge case companies we haven't considered.
I mean this is part of the problem I think right now and
why I think a data bricks IPO and the Mosaic deal will be
welcomed. It's like there's a public
market desperation for a way to bet on AI, which is why I think
NVIDIA has done so well because, you know, these big tech
companies are already very highly valued.
You know, the startups are private.
Cool, great. Thanks for coming on.
And now we're going to Chris Miller with Chip War, who's a
real genuine expert in the wonky world of chips.
And so we go deep in NVIDIA sort of their history and you know,
the, the big geopolitical question around the race with
China. So give it a listen.
Chris Miller, author of Chipwar. Thank you for coming on the
show. Thank you for having me.
I feel like so much of what sort of the regular person or even
you know, I don't know, the average startup founder, venture
capitalist in Silicon Valley thinks of this artificial
intelligence phenomenon starts and sometimes ends with chat GPE
3 and it's certainly it's very much the like.
What bots can we play around with?
Like how am I talking to some sort of AI but you know in the
background doing reporting on some of these conversations
NVIDIA and like H1 hundreds and like weird chips come up over
and over again and sort of people's ability to get access
to those chips. So I really wanted to have you
on just to sort of like interrogate that and understand
sort of the technology behind it.
Can we just start off like, when when does NVIDIA start to matter
as a company? Or just give us like, the brief
history of this company, because it was like a weird, weird sort
of gaming chip making company, right for a long time.
Yeah, that that's right. NVIDIA was founded over 3
decades ago, but its earliest couple of years was all in
graphics, and graphics was most important for games, computer
games, video games. Which, you know, think back two
decades, they had the most complex graphic demands, figures
moving across the scene, pictures changing very rapidly.
And there were special types of chips that were produced by
companies like NVIDIA for processing graphics.
And if you think back, not so long ago, people would buy
specific computers to have the best graphics capability.
Now we sort of take it for granted, but for a long time
that was a real differentiating factor.
Can you explain sort of the difference between AGPUA
Graphics Processing Unit and ACPU?
So it's ACPU, which is the the workhorse of traditional
computing and computers and data centers.
They're very good at doing many different types of things, but
they do every computation serially, 1 after the other.
And for most use cases that's what you want because you're
undertaking different types of calculations.
But for for training AI systems, you want to undertake the same
type of calculation repeatedly. And so parallel processing,
which is what a GPU does, is capable of doing multiple things
in parallel at once, which is why they're vastly faster for
this type of calculation than a CPU is.
But why? Why does AI?
Why do AI tests need to be parallel while like a normal
computer task is sequenced? Well, they don't have to be.
It's just really inefficient, right?
You know, I think the key trend, there's been great research
looking at the amount of data on which cutting edge AI systems
are trained and and what you find, if you look at this for
the last like 10 years or so and chart it, the amount of data
used in cutting edge systems is doubling every six to nine
months. There's extraordinary demands
for putting as much data as possible into training.
And in order to make sense of all this data, to train your
model to learn from the data, it has to undertake lots of
calculations. And so the the computing demands
of training have shot upwards not just exponentially, but
exponentially, at at a rate that's faster than Moore's Law.
And that's why having the right ship for training has become
very important. Did NVIDIA see this coming?
Like, you know, I their stock is up year to date.
I think it's some like 200%. I I feel like Jensen Huang, the
CEO, you hear about them all the time.
Like was this a happy accident? It's like we're we're making
great video games and all of a sudden people want to buy our
chips for the hottest thing in the world or did they have some
foresight here? How much was this a plan on the
part of NVIDIA? You know, it certainly was a a
bit of an accident and if you talk to the company they'll
they'll they'll literally admit that when they founded NVIDIA.
They had no idea that AI would be an application.
But over a decade ago, they began to realize that there were
a a bunch of PhD students and researchers at places like
Stanford and Berkeley who were using their chips for things
other than gaming for complex calculations.
And that NVIDIA realized that if they tried to build out chips
around this application and no less importantly, build out a
software ecosystem around it, that they could.
Provide the chips that AI would require.
And so they've been investing very heavily for now, over a
decade. And for a long time that
investment seemed actually quite foolish because most of their
money was still in in in graphics and in gaming.
It's only quite recently that a data center in AI has become
their their primary. Well, you just, people talk
about artificial intelligence as a way to boost their stock price
or to, you know, seem like a more futuristic company than you
are for NVIDIA. It's like, OK, you're a gaming
company guy. Like, chill out and now you know
it's been totally validated. That sort of all this talk of
artificial intelligence wasn't total wild hype, hypemanship or
whatever. Well, and in between though,
they were also a a Bitcoin mining.
Yeah, I know, yeah, exactly. They did have the crypto.
They're definitely good at seizing the moment, but they
were they were being used for crypto mining, right?
Is that what it was? That's right.
And like, yeah, how do you see Jensen the CEO, like he he's one
of the Co founders writer. What's sort of his role at the
founding and how do you see him as sort of ACEO today?
He was one of the three Co founders of a video they they
met in a Denny's in San Jose to devise the business plan that in
the the 1990s and and Jensen I I think deserves a lot of credit
for realizing. A decade ago, that AI could be a
real growth driver. At the time, people thought it
was a really irresponsible thing to do plow all this money into
building a software ecosystem around chips.
No one else did that. Most companies just made chips.
And if you go back to the debate in 20-15 around that time
period, you can find lots of equity analysts saying, you
know, what's this company doing spending all this money on
software rather than selling chips?
It's a waste of money And and worse, they were giving the
software away for free. But it turned out to be a
brilliant move because everyone started using the the CUDA
ecosystem, which is is the the software layer on top of NVIDIA
chips. And that's put it at the center
of the the AI world. A key piece of the chip world is
this sort of like are you designing chips, are you the
fab? Can you just sort of explain
that and where NVIDIA sort of fits in?
Like, are they? Are they actually we say build
the chips. Are they like building the
chips? Yeah that that's a that's a key
distinction. They're not actually building
the chips. They only do chip design from
day one. They were, they were what's
called a fabulous chip company. They didn't have a fab.
They only did the design and and chip design is in many ways like
a a software design type of business.
You you basically write the code of the design and then you
e-mail it off to the the factory or a fab and semiconductor
parlance and and the fab actually does the manufacturing
and so in the case of NVIDIA. Their most important
manufacturing partner has been TSMC, the Taiwan Semiconductor
Manufacturing Company, which manufacturers most of their
chips. And so today for all of Nvidia's
key AI training chips, they're all manufactured by one company
in Taiwan. And do they they make all
basically all NVIDIA chips or? They make, they currently make
all of the key AI chips that NVIDIA produces and they.
Been the most important producer of video chips from day one, and
is NVIDIA like, how far ahead is NVIDIA right now in this sort of
AI chip? Are they singular?
Like, yeah, where where are they today?
Well, in terms of market share, Nvidia's far, far ahead.
So it's estimated that up to 90% of cutting edge AI systems are
trained on NVIDIA chips. And you can debate what's the
definition of cutting edge, but Nvidia's market position is.
That is extraordinary right now and that's reflected in in in
their stock price. I I think if you asked are they
technologically ahead of of competitors, whether AMD, which
is another chip maker or Google, which has a chip called the TPU,
which is also used for AI training, that that's a harder
question to answer. Exactly.
Yeah, exactly, exactly. And that's a hard question to
answer because there's both, you know, technical specifications
you could look at, but there's also ease of use and so if
you've built a system. Trains it on NVIDIA if you
yourself as a engineer have gotten comfortable with
invidious products and with CUDA.
There's also a a a switching cost of moving to somebody else,
and so Nvidia's both got their own capabilities.
They also got the first mover advantage.
Everyone's used to their products, but but they're really
scarce right now, right? I mean, my sense talking to
startups and investors, I mean, you hear Nvidia's like in every
venture round now because as part of the funding, they can
sort of promise people their chips, which then they make
money back from the startups. It's like a great world.
They're playing kingmaker. Like what?
What's your sense of how scarce their artificial intelligence
chips are? Right now they're scarce.
I hear the same anecdotes that you're hearing and NVIDIA and
TSMC, the manufacturer partner have have both said it's going
to be a year or so until supply normalizes.
And the problem is actually not that TSMC can't produce enough
chips, but they can't package them because they use a a very
complex, a new packaging technology to put the chips
right next to the specialized. I assume you don't mean in a
box. Yeah, you're like, you place the
chip on something, right? Yeah.
That's right, yeah. It's a Traditionally chips were
packaged in. That'd be funny though, if it's
like. Oh, the Apple cardboard box is
like holding them up at the end anyway, so go ahead.
Yeah. So, so packaging traditionally
was a really boring part of the chip business.
You take a PIC a chip, you put it in a a plastic or a ceramic
package and that was that was done.
But now it's it's increasingly important because you need to
get your your processor and your memory as close as possible with
as fast and interconnect as possible.
And so companies like NVIDIA are leaning more heavily on having
the right packaging capabilities to provide the most advanced.
Outcomes. And so it's actually TSM, CS
packaging capabilities that are the limiting factor right now
for producing more NVIDIA GPUs. And and what is an H100 or like
what what, what are, how different are these chips from
the the GPUs that everybody got excited about in the 1st place?
Like are they server chips Are explaining, you know, what is an
H100? But it's just the newest version
of of AGPU that NVIDIA offers specifically for AI training
and. The reason people get excited
about new versions of chips is because the rate of change is so
fast that a new version is always not just incrementally
better, but a lot better than the old version.
You know, it's it's it's not like the iPhone 15, which is a
bit better than the iPhone 14. But we've stopped noticing and
I'm like, this is not enough necessarily to get me from 13
Pro or whatever. Yeah, yeah.
The chips are different because of the the Moore's Law dynamic,
which is, you know, says you roughly double your processing
power every two years. And that's not a perfect, a
perfect metric, but ballpark that tells you just how rapidly
the technology improves. And the H100 is the most
advanced chip that NVIDIA offers.
So yeah, to put Moore's Law is still alive today.
Moores law is actually not a law.
It's a prediction that was set out by Gordon Moore, who founded
Intel in 1965. He predicted that the number of
transistors per chip would double every year or two, which
means that the computing power of chips would double every year
or two. And that's been and he said it
would be a decade, but then it's been true for a long time,
right? That's right, I'm just stealing
from your book. Of course it makes me sound
smarter when you get to read the book in advance, so you know all
the answers. Anyway, go ahead.
But yeah, that's right. So Gordon Moore thought, would
last for a decade, and here we are half a century later.
And Moore's law is is still basically holding up the the
rate of change increases and decreases at certain times.
But it's basically true that if you wait 2 years, a new chip
will have twice as many transistors and therefore twice
as much computing power. I mean, this is sort of a hard
question, so I don't know if you there's an easy answer, but
yeah, how much credit would you give then the improvement of
GPUs for what's going on in artificial intelligence right
now relative to, you know, attention is all you need or you
know, the papers in terms of the approaches to actually use these
chips to to run, run the software?
Well, you know, I think it's certainly it's the case that
that there's there's multiple factors driving innovation.
But I just the way I like to think about it is, you know
what's improved most over the last decade?
Is it the case that that software engineers are 16 times
smarter or is it the case that? Algorithms are 16 times better
or is it the case that because of Moore's Law 2 to the fifth,
we've got, you know, a vast increase in the number of of
transistors? You know, I think it's it's
primarily computing power that's that's driving us.
That's what you want. The chips guy comes on, he says
it's the chips. Yeah.
A key theme, you know, obviously in the book and sort of for
anyone thinking about the situation is, you know, the
rivalry with China and and sort of the the delicacy of the
global supply chains and you know, the, there's the chip
rivalry with China. But also in artificial
intelligence, you know, if we're worried that this is the
potential sort of path to some sort of, you know, generalized
intelligence, there's also the real question of whether the US
gets there first. Yeah.
So I mean, there's so much to hit at.
I mean, I guess the first question for me is, is the
United States getting most of Nvidia's chips or like how much?
How many of those chips are going to China?
Is China getting access to those right now while they're scarce?
Well, the the US last year imposed new rules that said
NVIDIA can't transfer its most advanced GPU to China.
And not just NVIDIA. Any company producing chips
above a certain threshold can't transfer them to China, so H1
hundreds are illegal to. To transfer to China, China's
not getting any of those. But but NVIDIA has design chips
that go basically right up to the threshold called A8
hundreds. And Chinese firms are reportedly
buying very large volumes of those.
So China's getting a lot of GPUs, but they're less advanced
than what AUS firm can buy. What would their next best
option be? Well, the next best option is
Nvidia's second best chip. Oh, man, OK, right.
And China tried to build its own fab, right?
A competitor of TSMC. What's, what's the status of
that and how much? You know it's fun, especially in
the US and given the market returns, like to talk about
NVIDIA, but like how much does this all hang on TSMC?
So China has been investing very heavily in its own chip
industry. It's got both companies doing
the design side and the manufacturing side.
There there are a number of Chinese GPU designers that seem
to have pretty competitive products, although even in China
Chinese firms prefer to buy invidious chips, so they're
they're not as good it seems. But the big challenge in China
is the manufacturing side because TSMC, the Taiwanese
firm, has been around five years ahead of the leading Chinese
firm SMEC. For a very long time, for at
least a decade, there's been a five year gap.
So both companies regular regularly improve the
manufacturing processes, but the rate of change in both is
roughly the same. So TSMC is always half a decade
ahead, and that holds to up the Today.
And what about the United States?
Like it hadn't Intel been trying to become a fab?
Like what's what's the state of our ability to actually
manufacture these chips if you know, there's a war in Taiwan,
God forbid? Well, yeah, that that's that's
the dilemma. So Intel does make its own
chips, but around five years ago it faced some severe problems
with its manufacturing operations.
And so for its most cutting edge chips today, Intel now turns to
TSMC for the manufacturing. Now Intel right now is is trying
to reformulate its manufacturing processes and it hopes that by
they're saying by 2025 they're going to be producing.
Have their most advanced chips back in house again, and that
those chips will be as capable as a TSMC made chip.
But right now, Intel's most advanced chips are actually
manufactured by TSMC. And then there's like a key, the
people who make the machines that help T that allow TSMC to
do what they do, There's like one company of those or what's
that company? Yeah, that that's right.
So TSMC, they know how to use the machines to make chips, but
the machines themselves are produced by a handful of other
companies, a couple in California, a couple in Japan,
and then one in the Netherlands called ASML, which produces the
most complex of these chip making tools.
So is quantity. Everything here with the H1?
Hundreds like, I mean, you know, I'm at, you know, Microsoft has
this cloud computing offering. You know, Google has its, Amazon
obviously has Amazon Web Services.
How much are those services trying to stockpile AI chips?
And you know, then these startups, whether it's, you
know, anthropic or open AI, how much is their game?
Just like we're going to win if we assemble the most H1
hundreds, Is quantity sort of the game right now in terms of
getting access to these chips? So long as there are shortages,
I think Quantity's a key part of the game.
And and what you see is that NVIDIA now realizes it's got a
lot of influence over the future of the cloud computing market.
And so NVIDIA has been, I I think, pretty actively trying to
build up other cloud computing firms by giving them exactly
exactly giving them access to GPUs and NVIDIA.
I think quite rationally wants a more fragmented cloud computing
market because that's a market in which it has more market
power relative to Microsoft, to Amazon and to Google.
So so we alluded to this, but sort of can you walk me through
the tech giants in terms of credible competitors to NVIDIA?
I mean we talked, I guess start with Google, it sounds like
maybe they're in the lead. I want to know about Intel.
And then is there anybody else who's who's relevant or even
close? Yeah.
So Google has a a in house designed chip.
They've got a big design team that has designed a chip called
ATPU which stands for tensor processing unit but fairly
similar to AGPU that is used for for AI applications and in
Google has a vast cloud computing business as well as
its own, its its own vast data center.
So lots of experience running very complex computing
operations but empirically we know that companies prefer to
use NVIDIA chips at least they do right now.
You know that could change as Google's chips get better as
scarcity drives up the price of GPUs.
But that hasn't been the the trend, the preference has been
for NVIDIA. Then AMD is the other company
that that is a competitor that in producing GPUs it's a a chip
design firm. They also manufacture with TSMC
that in. They're an American firm.
That's right. Based in Texas and and they
manufacture fairly competitive GPUs.
But the key difference with between their chips and Nvidia's
is that NVIDIA has the the ecosystem around it, the CUDA
ecosystem. And so once you're bought into
that ecosystem, right now the switching costs are substantial
enough where you just prefer to stick with NVIDIA if you can get
access to enough chips. Is Intel relevant at all?
Intel is is trying hard to produce competitive chips and I
think over the next couple of years we'll see it roll out a
series of new products that are designed to compete.
But right now it's a small player.
In my understanding like in cloud computing, like part of
the beauty of like an AWS is that you can sort of you know,
it's flexible, it's elastic, we can sort of have some people do
the computing, then you sell it to another customer at a
different time. Why hasn't that been the case
with this artificial intelligence computing, I mean?
I hear people talk about it like you're you're running full bore,
like these data centers are super hot.
Like what's what's so different about AI processing?
I think the the, the key difference is that the demand
for compute power in AI training is so large that there's just a
deficit of it. There's just a deficit.
And so the the systems that firms like Open AI are trying to
train are so vast that they they can't share their infrastructure
because they need all of their infrastructure and they need
even more of it than they actually can get access to.
And so the cloud computing business has been a business, as
you say, about engineering more efficient systems via sharing.
But in AI training, no one wants to share because everyone needs
more compute than they can actually get access to.
Right. And then there is also this sort
of perceived almost like 0 sum, like you want to deny your
competitor access. One company we haven't talked
about, you know, that we talked about in the laptop world for
chips is Apple. Like what's what's the status of
Apple on this? Sometimes people talk about
Apple and artificial intelligence on like local
processing. Is that a possibility or how do
you see Apple as being relevant or not in this race?
Yeah, you know, thus far we've been talking about AI training
and and AI training happens almost exclusively in big data
centers. But there's a big question about
what inference will look like in the future.
Some people think that inference will mostly happen in data
centers, that you're gonna ask a question of ChatGPT, you'll send
it back the Internet to the data center, tips to the data center.
We'll think about it and give you an answer back.
And that is efficient in some ways.
You have all the compute, all the chips in one big data
center. And so you can start to set up
some of the sharing and efficiencies that that we just
discussed. But the downside is you have
latency issues if you're sending all the data back and forth and
there's costs associated with moving data.
And so an alternative paradigm is doing more of your inference
on the edge of networks in your phone, in your car, for
autonomous driving systems, in in in your PC.
And that's where companies like Apple could start to play a much
bigger role. Right now, I think the the
market for inference on the edge is still very much in flux.
We're in the process of seeing many new types of chips rolled
out precisely for that purpose. So, so it's just sort of depends
on where it's one of these like sort of personal computer versus
server level questions back in the 90s where it's like OK, is
computing going to continue to happen?
In the cloud? Or is there some model where
local computers are going to do and you're saying that if it is
local, then maybe Apple's in a better position because they
have these great chips for for individual computers.
Is is that right? Yeah, that that, that's right.
And there's I think a pretty straightforward cost equation
of, you know, if the cost of data transfer is high, you'll
try to do as much as your processing can on the edge.
If the cost of the transfer is low and the benefits of having
big centralized facilities are are substantial in terms of the
efficiencies you can reap, then you'll have a much more
centralized inference process. And it it might vary even
between different use cases. In a car, for example, if you've
got a pretty autonomous car, your willingness to to depend on
a data center to tell you to turn left or to turn right, it's
going to be pretty limited. I would, I would, I would
hypothesize. Whereas for your phone maybe.
You want to make sure no matter what you can get the answer but
you know if it's for mid journey I can wait till exactly you
know, I mean we we talked about NVIDIA earlier on in terms of oh
there was a brief period where they were essential to crypto
mining and I and one of the key storylines I think think for the
non crypto world that was just watching it.
Was all the energy consumption around those crypto mines is?
Is that true for AII haven't heard that same environmental
story get picked up in this case.
But are are we destroying the environment and our pursuit of
artificial intelligence? Why, I think we we haven't heard
much about it just because there's such an extraordinary
demand for computing that companies haven't gotten around
to thinking about the consequences.
But the short answer is, is yes, it's extraordinarily energy
intensive and and we're going to get more efficient at it as as
time passes. But right now there are, you
know, such demands for energy in data centers that there are some
places in the US where it's difficult to build a data center
because there's no spare electricity capacity for the
data center. It's it's already a gating
factor. Have you seen any?
Yes, it's about energy use or anybody trying.
Well it's it's, it's tricky because everything depends on
how much more efficient you think things will get over time.
So you've got to assume we're going to be a lot more efficient
making the same compute available at lower energy demand
in 10 years time. But at what curve it's it's very
difficult to say. Yeah.
And to be clear, I think what what is the point of energy if
not to sort of push the cutting edge of like computing?
It seems like a very valuable use to me, but it yeah, it'd be
interesting. I mean, you obviously want to
make sure companies aren't being sort of cavalier about it.
And yeah, well, there's certainly not cavalier because
if you look at the cost of running a data center, 'cause I
don't know what it is, yeah, critical cost.
Well, I mean, I don't know if you're Open AI, you raised $10
billion, it's you can become sort of irresponsible.
You know there isn't the same market pressure if these
companies are able to raise what are basically like research
grants to to figure this all out.
Can you, I mean, we sort of jumped into artificial
intelligence, but I mean can the broad theme of the book, the
idea that sort of that we often think of global infrastructure
and the sort of weaknesses and vulnerabilities in terms of like
oil, right. I mean it's like oh, did we go
to war in Iraq over oil? What what is sort of the real
sort of what are the weaknesses with chips and what what why do
you think sort of this chip's arms race is?
So important when we're thinking about sort of geopolitics.
Well, the reason I, I, I wrote chipwork was when I realized
first, that we're surrounded by thousands and thousands of chips
over the course of our daily lives, Not just our phones and
our PCs, but it's cars and dishwashers and coffee makers
too. Second, when I began to
understand how critical they are for AI.
And then third, when I realized how concentrated the production
is in just a tiny number of countries and really just a
handful of companies, Taiwan being the the, the, the, the
most surprising example of that. And so when you look at the the
US, China race to develop more advanced AI systems and in tech
in general, then realize that both China and the US are
dependent to a shocking degree on ships made in Taiwan to power
their most advanced AI systems. It's an extraordinary situation
that we've all found ourselves in and it's a very dangerous one
given the concentration and given the dependency.
What would be the consequences of China having a chip edge
broadly and an artificial intelligence?
Well, just like we're preventing China from accessing our most
advanced ships, I think one has to assume that any country with
that position would do the same to us.
And right now, Chinese firms that are trying to develop AI
systems are doing so at least severe disadvantage because
they've got much less access to high quality hardware than does
Open AI or or Google. And that's I think the only way
to accelerate into the future as countries, including the US
become more restrictive in terms of controlling access to
different types of AI hardware. And so it's a good thing that
we've got most of it designed here and most of it manufactured
in friendly countries, because what we're going to see, what
we're already seeing, what we'll keep seeing is more
politicization, I think, of the hardware that makes AI possible.
Is the United? I mean the United States still
seems great. I guess that chip design, given
we have so many of the leading firms, could, do you think we
could ever produce a fab or like what?
What makes TSMC so uniquely capable at building a fab?
Like, we're talking culture, money, like, what's what's
driving that in your opinion? Yeah.
You know it's it's it's not it can't be a cultural thing
because Taiwan hasn't always been the center of the chip
industry. And if you go back four decades
ago, it was Japan that was the world's biggest producer of of
advanced semiconductors. So it's primarily I think the
result of TSMC business model which has let it scale and the
scale has let it Dr. efficiencies.
And so when TSMC was founded in 1987 at that time almost all
companies both designed and manufactured chips in house.
They did, both sides of the of the equation, And the founder of
TSMC, Morris Chang, realized that as it was getting more
complex to manufacture, companies would prefer to
outsource it. And so he decided that he wanted
to be sort of like Gutenberg was for books.
Gutenberg didn't write any books, he only printed them.
Morris Chang and TSMC, they didn't design any chips.
They only manufactured them. And what that meant is that he
could produce chips not just for one company, but for lots of
companies. For AMD, for Apple, for
Qualcomm, for NVIDIA. And so today, TSMC is the
world's largest chip maker. And precisely because it's the
largest, it's also the most advanced because it can hone, it
processes over every single silicon wafer that it
manufacturers. How can you can you make any
predictions about how much you think AI chips are going to
improve? Or like what do you see sort of
the next couple years looking like in terms of like do
availability problems resolve and and how much more do you
think these chips will improve over the next couple years?
Yeah, on the the availability front, NVIDIA and TSMC has said
they expect it to be a year or so until the shortages get
resolved, but I think they're going to get resolved.
Prices are such that there's a very strong incentive to solve
the availability problem and sell more.
Sometimes people think they want to keep a shortage to keep sort
of demand high. You you think they want to
produce as many chips as they can get out there to make the
money while it's there to be made or yeah I think that's
right because they're they're it looks like they're a
monopolistic producer. They've got 90% of the market
but if they can't supply chips there are competitors there is
Google with TPUS there is AMD. And so if there are long run
shortages that that are are in the supply chain, people will
turn to competitors. And so they've got a strong
incentive to get get supply increase.
And actually part of their strategy is again to put their
chips at the center of the AI world and make it the the gold
standard. And the more people that design,
the more people that train, more people that study using their
system using the Kuda ecosystem, the more likely NVIDIA is to be
at the center going forward. And then I guess sort of the
next chip they have is hard to know or have there been any
leaks about what what the future looks like in terms of these
artificial intelligence chips? I mean I think the the best
guide is is to look at the past and and what you find is that
that for over the past half decade they've released a series
of better and better chips that you know ballpark Moore's law
gives you a pretty good guide as to where things should be in a
couple of years time bigger, more powerful hopefully somewhat
more power efficient and and the other area focuses the the
interconnect speed between semiconductors.
So there's the GPUs themselves, there's all the networking
equipment that moves data between chips.
That's also an area of really intense focus right now.
And we're I think we're gonna see a lot of progress for the
coming years. Jensen I mean, did you interact
with him at all for this book or no, I didn't actually.
I I interviewed his his Co founder but not not.
Interesting. Yeah.
I mean, just like observe. Like it must be crazy to go from
sort of being seen as this, like, I don't know, wild sort of
futurist to all of it. Seeming to come true or the
world sort of agreeing with your futurism.
I mean he sort of he he wears like leather jackets and dresses
all black or like what? What do you make of him?
And yeah, how much has this been sort of a vindication for him?
Well, I think he he sort of sees himself like like Steve Jobs was
for the iPhone moment, he is for the AI moment.
And I think that's right. I I think he he has played that
role because he does with a lot of credit for investing in the
types of products that that have made AI possible at a time when
most people thought it was either crazy or possible but
only relevant for niche academic uses or something that would
never be financially viable. And look, he's built one of the
world's largest tech companies around it.
And already there are products being produced on a regular
basis using chips that his company has pioneered.
I believe I agree with you. You can make a fortune.
Silicon Valley can love you. But I feel like the world will
only love you if it's your device in their hands or it's
your brand. You know, it.
I, I, it just doesn't seem like they have a path for like
consumer love, right? I mean, do you see them trying
to take advantage of this and go, you know, offer other
products or, you know, they're so good at chips, they're
they're gonna, they'll win the technologist's heart and be
happy with their cash mile. Yeah.
They're so far from the computer in terms of where they sit in
the tech stack that like most consumers don't even know they
exist. Or gamers do is sort of the
funny gamers do. Right.
Yeah. How did how did you get into
this? Like what were you sorry, I
don't know as much of your background.
Yeah. Before you wrote this book, were
you a chip nerd or what? No, not at all.
Oh, really? OK what was your story to the
book? So I I'm a an economic historian
by training written a couple of books on different aspects of
economic history and and decided to write the book when I came to
realize how important ships were and was shocked myself that I'd
never paid any attention to this industry.
And so it was the combination of learning that almost all
advanced ships are made in Taiwan, which was shocking.
Learning that chips are so extraordinarily difficult to
manufacture and transistors in your iPhone, for example, are
small in the size of a coronavirus manufactured by the
billions. That was an extraordinary fact
that got me into it. And then and then finally
looking at the ramifications for AII, like most people have
thought about AI as a as a question of of of either
software or of data. Data is the new oil I'd read a
million times, but it turns out that's not exactly right.
Did you go dump all your money into NVIDIA or I should have?
Yeah. Did you see the stock?
I mean I I know. Well you're if you're an
economic historian like do you have a view on whether, OK, this
is a great company but it's overvalued.
I mean you know, yeah it's do you, I mean you shouldn't turn
to me for for stock market advice.
But I mean I think the, the key question about NVIDIA is, is
will it. I, I think it's clear it will
continue to play a dominant role in AI training.
Maybe it'll lose market share, maybe it'll win market share
with Google, but it'll play a central role.
But in the inference market will NVIDIA be as central there or
not? That's I think the key question
that we're gonna find the answer to over the next couple of
years. And do you think China is close
to catching up? You know, I I think China's is
not that far behind, but China has been not that far behind for
some time. And because the rate of advance
is so rapid in terms of design in the US and manufacturing in
Taiwan, it's very, very hard to catch up.
And so long as that companies like NVIDIA and TSMC keep racing
forward to catching up is just going to be extraordinarily
challenging for China and no matter how much money they pour
into it. And are Japan and South Korea
relevant to this conversation? Well, it's that's an interesting
question. Japan is relevant because if you
look inside of a chip making facility, there is a ton of
Japanese machine tools and there's also a wide variety of
ultra specialized chemicals made in Japan.
So a lot of the chemicals used in chip making because of the
precision required have to be purified to the 99.99999% level.
A lot of those are only made in Japan.
I wanted to go back and I know this is like a complicated
question, but like what are the tasks that are being thrown at
these chips, right? Or you talked about sort of
matrix computing or? Is there like 1 task over and
over again that these chips are being asked to do or it's a lot
of different things or how? How are the foundation models
sort of translating to the chips as best you can explain it?
I I guess the way I would explain it is you know if you're
trying to train a a a chat bot like ChatGPT, you train it by
taking all of the language data in a a vast data set like
Wikipedia. You essentially read Wikipedia
and identify patterns, and so and.
That's and that's like inference that you've been taught.
That's training. That's training.
So training is is identifying the pattern.
So you know the what's the what? What is the next word I'm going
to say? You know, you know what say, and
Chachi Piti knows that because it's identified thousands of
other instances where sentences like that ended with the word
say. And it did that because it read
lots and lots of books and articles and Reddit posts
online. And then when I you go back and
ask a question which is inference, it gives you a pretty
good answer because it has a pretty good sense of what the
most likely word is going to be. So you're saying they're they're
training the chips based on a a lot of data and then it makes an
inference about what it thinks the the sort of fill in the
blank of of the prompt? Basically, yeah, inference is
just an educated guess, a probabilistic guess.
And what's cool about Chachi PT is that it's not just fill in
one word, it's fill in dozens of.
Words. I know, it's insane.
One thing I'm still trying to get my head around is, you know,
like DeepMind, right? Like when they trained, you
know, computer systems, I guess to to compete on chess or Go or
Dota. Like my sense is that you know
they're using. Sort of this sort of scale data
approach, but they also have like specific ways they're
teaching it to think about specific problems.
Or like, do you, are those types of approaches to AI also using
these same chips, or are they still sort of invested in the
same arms race? Yeah, well, the chips just do
the math for you. And so they'll do what other
math? Whatever math you ask them to
do, Yeah. Exactly.
And they're not. And then the interaction between
like Nvidia's like programming language is super interesting
because they're not like the the foundation models are not being
written in that language, right? But they're they're sort of
bridging. That's right.
Yep. And it's it's just about, it's
fundamentally allows them to optimize what's happening on
these chips much more than maybe they would on like an AMD chip
even if it were and it's it's it's it's easier, it's more
familiar and therefore there's a big Moat around invidious
position. Yeah.
So I have you like become like were you bullish on like AI
before writing this book or I'm curious how much your view has
really changed on, yeah like I don't know like generalized
intelligence or like the imminence of self driving cars
or any of that. You know I I think that actually
we've I I'm I'm in the long run, I'm very bullish but I think in
in the the short run the last 12 months there's been all a whole
lot of excitement of the imminent arrival of AGI which
I'm I'm not so sure I see it happening in 2023.
I think if you look at the trends, the trends are all very
positive. AI system is getting better,
more capable, but I think there there there was due to ChatGPT a
popular sense that we're going to get some extraordinary
products in our hands right away.
And actually, it's going to be a couple of years before we see
AI, the current wave of AI products deployed in a big way
that meaningfully impacts economy or society.
Yeah, I think we're all watching to see if this hype wave is
about to end or not. I mean there's there's a world
where this is sort of peak NVIDIA, right?
Like we have this sort of mania around AI, everybody's running
towards it and then it turns out, you know, chat GPD doesn't
improve that much and self driving cars still have a lot of
edge cases. And you know, the chips don't
solve everything. Even if they do get better.
Like you're saying, like, I don't know.
How likely do you think that is? Or do you have a sense of, yeah,
whether better years are ahead or this is sort of a peak moment
for the company. Well it's it's been a very, very
good year for the company. So I wouldn't be surprised if
it's a it's it's at least a localized peak.
But I I think you know for for AI applications most people
think of what's the consumer application for AI, you know
what's what's the iPhone of AI. But it seems to me that for the
next couple of years actually most of the application and most
of the money in AI will be in enterprises because enterprises
that have vast data, they've got the desire to use it efficiently
to monetize it. And so that's where I think
we're actually already seeing a lot of the investment and a lot
of the early products are happening in pretty boring
places inside of enterprises. But that that's probably OK
that's that's what's going to drive I think the the, the
building of effective products in the long run.
Enterprises, you know, businesses just translate it for
like the regular person, you know, I used to work at
Bloomberg. They've actually come out, I
think with their own foundation model to show how they can train
financial data. And you you have a good example
of where you think corporations will put this approach and data
to use. If you imagine a company like
Walmart, they have to everyday decide what price to put all of
their products, and they decide pricing based on a whole variety
of different factors. What their competitors are
pricing? The availability of products.
Very complex process and that seems like a perfect use case
for trying to use more advanced AI algorithms to set better
prices more rapidly. Boring is it?
Kmart this blue light discounts. What's Walmart?
I forget. But yeah, who gets the discount?
You know where, right? A lot of products to keep track
of. Have you like?
It feels so absurd to ask this, but like, what the chip business
looks like in a world of AGI, generalized intelligence?
Or like, have you gained that out at all?
I mean, there are people who think it could happen, you know,
five years from now or less. Like, do you have a view on it
first of all and then second? Yeah, what?
What would it mean for the chip world?
Well I I guess my sense is that we're we're we're a long way
away. I'm I'm still waiting for
ChatGPT to accurately complete all my senses.
You know still at a 90% rate and the other 10% are pretty bad.
I think it's gonna require tremendous advances in
semiconductors to make possible the tremendous increase in
computing that more advanced AI systems will need.
I mean that that's been the trend.
The trend has been AI systems only advance.
We apply more computing to them, and so if we want systems to be
twice as good as they are today, we're gonna need something not
too far off from twice as much compute to make it possible.
Are there, you know there was this I I covered Righetti and
sort of the quantum computing world.
Do you have much optimism there? Do you spend time on them in in
the book? You know, I not not in the book.
I my sense in in, in, in speaking to people in the
quantum world and then looking also at how in history new
computing technologies have disseminated is that actually
even revolutionary technologies, they're implemented slowly.
And so suppose we get to the next couple of years, the first
practical use case of quantum computing.
It's going to be a years long process of beginning to
implement that in all sorts of different computing use cases.
And so I think we shouldn't expect to have, you know, a
quantum powered iPhone anytime soon.
It's it's insane to think like on the one hand we have.
We have public companies that people are sort of speculating
and betting on they yeah produce quantum computers but they
really do nothing practical. I I you can go with I you know I
swing back and forth. On the one hand it's like great
it's amazing that our system will invest in such bleeding
edge technology and give it a chance.
On the other hand for the shareholders, you know like
everybody could decide to give up on the effort with like
higher interest rates, it's, I don't know, it's a crazy
function of. The global economy, well, I I
think it's interesting that a lot of the companies that are
investing the heaviest are also big cloud computing operators.
And that's I think because they believe that quantum will be
actually most useful in a context where it's closely
interlinked with huge volumes of classical computing.
And so actually, we're going to need more advanced silicon chips
to make quantum practically applicable.
And do you know, is there any sense whether Quantum and the
GPU sync up, or where they fit in that story?
There There are many different paradigms for how you
specifically sync classical and quantum computing, but right now
nobody knows which of the many paradigms will win, if any of
them. To wrap it up, I mean, what do
you think the regular person should take from all this?
Like it feels very far. From their lives like they they
maybe don't even know what NVIDIA is.
Like they tried ChatGPT, their kids using it for homework.
It's not in their life yet. Like you know Silicon Valley
loves to get itself worked up about things that don't
sometimes translate. Don't like what what do you
think the lesson or the the key thing for like the regular
person right now is in terms of what's happening with these
chips. I think if you went back to
1965, which is when Gordon Moore first set out the the phrase
Moore's Law and you asked what's the impact of of of Moore's Law,
the average person, the the answer in the short run was
approximately nothing. But the answer in the long run
is that it totally transformed society, economy, technology,
everything, because we put computing and therefore
semiconductors into basically every product that we rely on.
And I think we should expect the same to be true for AI.
You know, what does it mean for me tomorrow?
Probably not much. And what is it going to mean in
in 10 years and in 20 years when every aspect of human life is
being impacted by it? Well, it'll be transformative in
all sorts of ways, most of which we probably can't even imagine
today. For the startup entrepreneur,
where do you think there's opportunity to build a business?
You know they, I mean maybe you think they should go out and
start the next NVIDIA like it's a heavy lift, but like where
where do you think really with the progress that we've made in
these chips that there's real business opportunities still?
Well, I think if you've got the idea of the next NVIDIA, you
should absolutely go do it. But I I think you're, you're
right that, you know, companies like NVIDIA, they they create
the infrastructure on which many different types of systems can
be built. And and if you go back to, you
know, the smartphone for example, smartphones were
themselves a platform owners could build lots of different
things. And I think we're still in the
Super early stages. We're basically in stage zero in
terms of figuring out what are the ways you can create products
out of generative AI systems. Right now, there are hardly any
companies that make money selling their AI systems.
And of course, the challenge for entrepreneurs is, you know,
people who built early iPhone apps and Facebook apps, you
know, they don't all pan out. Sometimes you come too early.
You can be right that a technology is transformational
and get the timing wrong. And I guess that's.
The sort of confluence of luck and insight in, in the business
world, it's a tough one to predict.
Chris, this was awesome. Thank you so much for coming on
the show. It's great.
Great to talk to you. Thanks for having me.
That's our episode on chips and big tech.
Thanks for listening. I'm Eric Newcomer, your host,
author of newcomer. Thanks so much to Max Child and
James Wilsterman. Co founder Zavali and my long
time friends shout out to Scott Brody, our producer Riley
Kinsella, my Chief of Staff Gabby Caliendo at Volley who's
helping organize the conference and playing a big role behind
the scenes. Thank you to young Chomsky for
the theme music. Please like, comment, subscribe
on YouTube, give me a review on Apple podcast and subscribe to
the sub stack newcomer.co. Go try out Volley on your Alexa
play song quiz maybe we're going to have.
Couple more episodes before Cerebral Valley.
Cerebral Valley is on November 15th.
We will publish our conversations to YouTube.
We will probably publish them all.
That's what we did last time. You can see the conversations
from Cerebral Valley One, which is on March 30th.
And then yeah, we'll play some of our favourites probably.
We did a sort of distilled version in our podcast feed, so
follow the podcast feeds for those.
And yeah, I'll be covering it in the newcomer newsletter.
So newcomer.co, Thanks so much.