Everything's nice with AI until it makes a mistake that a human
would never make. Ever.
And this reveals that it wasn't actually smart.
You can't really eliminate hallucinations.
They happen when you least expect them.
It happened to Air Canada. The chatbot gave the wrong
policy to a customer. And then Air Canada had to honor
the policy because the judge told it to.
Emmanuel Maggiori is AI industry insider, who has developed AI
for a variety of applications. He is the author of the books
Smart Until It's Dumb, Siliconned, and The AI
Pocketbook. In your book, you mentioned
about two AI booms that happened.
But actually it ended up going nowhere in the end.
So maybe tell us a little bit of this history.
There was an AI boom in the sixties.
Everybody will lose their jobs in 20 years.
That was said in the sixties. Then in the eighties, another
boom, another collapse. There are concerns, what will
happen to my job? Especially for tech, software
engineers, many that can be replaced, including juniors.
What is your opinion? There will be jobs that will be
lost to AI, for sure. Maybe prematurely sometimes.
We've already seen companies firing people and now hiring
them back because it didn't work.
Business is scrambling so hard to adopt AI.
How can business start thinking about adopting AI?
Building a successful business is a business problem.
It's not a technology problem. I've seen so many projects fail
almost the same way every time. The best way to use AI is to
acknowledge these hallucinations from the get go.
If you embed that into your product, you can build a
successful product. Then they tell you a AGI is
coming, right? I would tell you the answer is
no. AGI is not coming anytime soon.
And the reason is that... Hello, everyone.
Welcome back to another new episode of the Tech Lead Journal
podcast. Today, I have with me someone
very excited, to, you know, share some learnings or some
insights that he has been dealing with regarding AI.
I know maybe this is like on everyone's mind these days,
right? Emmanuel Maggiori actually wrote
two books related to AI. The first one is called Smart
Until It's Dumb. And the second book is titled
The AI Pocket Book. I find this a rare opportunity
for us to learn more about AI. What actually these days, you
know, general- generative AI, LLM, all these tools that we
have been hearing a lot and using a lot in our day-to-day
work, try to understand the basics and maybe few interesting
things that Emmanuel can share today for us to learn from him.
So Emmanuel, thank you so much for this opportunity.
Welcome to the show. Thank you for having me.
Right. Emmanuel.
Uh, I always love to invite my guest first to maybe share a
little bit more about you by sharing any career turning
points that you think we can learn from you.
Career turning points? Yeah, I think there were two
important ones. The first one was when I
realized that a lot of the technology industry was driven
by hype. You know, a lot of projects were
just, I don't wanna say fake, but a little bit fake.
Sometimes, not on purpose. But you find yourself working on
products or on building products that don't make any sense and
everybody knows it, right? There's a difference between an
experiment that is worth doing, right?
Okay, we don't know how the client will react to this, so
we're still going to build it as a sort of MVP.
There's a difference between, you know, that, and just
building something that doesn't make any sense.
Everybody knows it, but you're still building it because a VC
told you that it was a good idea, or because an upper
manager from a company just gave you a lot of money to do this
and then you're just doing it. And unfortunately, I found
myself involved in projects like that a little too much.
And at one point, I thought it seems like most of it,
especially when you work on cutting edge stuff, it's like
that. And I think it was a turning
point because I was very naive before, right?
I thought companies are efficient.
You know, why would a company ever carry out a project this
long when it's obvious that it will never work.
But you know, they do. Things like that can last for a
very long time and these projects, um, they get swept
under the rug very easily. So I think that was an important
career turning point. And I think another one, which,
you know, well, that first career turning point led me to
the book Smart Until It's Dumb, what you mentioned, right?
It was, I need to tell the world that AI is not that great.
That there are lots of problems with it.
That lots of projects fail. And why?
So that was why I wrote that book, right?
And then another turning point was, you know, I thought I'm
gonna work on more foundational stuff, less hyped up stuff.
Just do some more traditional software engineering.
But then I realized that a lot of people weren't working much.
They weren't doing anything, right, including me.
I found it hard to find work to do.
I was hired by a company and I thought it was going to be super
exciting and they kind of put me in this staging area to wait
until I was assigned to a project.
And there were lots of us in there being paid to do nothing,
right? And that, you know, I thought
something's really strange with this field.
It seems like there's a lot of money flowing into the field,
lots of projects. They get launched without really
knowing what, you know, they're gonna be or what we're gonna do.
And companies like to hoard staff sometimes.
And I thought, again, this is companies not being very
efficient and I didn't think it would last.
And then everybody got fired, right?
Twitter fired 80% of its employees and everybody.
You know, Meta, all major companies, made a lot of people
redundant. And I thought, you know.
I mean, some people work very hard, but a lot of people, they,
they're just, they just don't know what to do.
When I thought that wasn't sustainable.
So that was another learning for me.
Wow! Very interesting sharing that
you just shared just now, right? So I find, I mean, when you said
that I also had experience, uh, of those kind of, you know,
projects, career, right, where, you know, seems things are going
to the wrong direction, but we still march on anyway.
We think that something positive will come out of it.
And the second one, you know, especially for consulting or
professional service kind of work, sometimes, yeah, we were
put on the bench or on the beach, some people also call it.
And maybe in some big organizations, there are roles
which have maybe little things to do and, you know, spend their
time doing, you know, something else which is not necessarily
productive. So thanks for sharing that
personal journey that you have so far.
And I know the books that you've written is something about AI,
right? So probably you can share a
little bit of background why you wrote those two books.
Like what kind of things that you actually want to convey by
writing these two books. So I think my first book came
out of this frustration of everybody saying that AI was the
future. We should be AI first companies.
And this was, look, I wrote the book before the ChatGPT thing
happened. But ChatGPT was released when I
was finishing the book. So I added stuff about it, but
it was kind of about that time. But what I'm trying to tell you
is that I've been noticing this before, even that wave of AI
hype that we witnessed. And I wanted to tell people how
AI actually worked, you know, how machine learning works and
why it has some issues, let's say.
Um, there were lots of, you know, it was a very broad book.
I even spoke about consciousness or the mind, you know, because
you have all these people even telling you that we're just
building artificial neurons. But in reality, we don't really
even know how neurons work very well.
So how can we actually be building them?
So I thought it was, it's good PR, but it wasn't quite true.
And I wanted to also convey, if you want my experience in the
business of AI, because I've seen so many projects fail that
identified a way in which they fail.
Almost the same way every time, repeatedly the same mistakes.
And I thought, I need to tell this to the world, right?
So it was a kind of an antidote to AI hype.
That was my, the goal of my book.
Right. So, yeah, I remember back then,
I dunno, like 5, 10 years, uh, ago, right?
So there are other fields of AI compared to what we know now,
which is like generative AI, LLM, you know, all the craze
about these kind of AI. But in the past, there are so
many AI technologies as well. And in fact, in your book you
mentioned about two AI booms that actually happened, but
actually it ended up going nowhere in the end.
So maybe tell us a little bit of this history so that people know
about the context and they can relate to what they're
experiencing now. Yeah, so there was an AI boom in
the sixties, which is kind of crazy because computers were
new, right? And they weren't very powerful.
But people were saying all these things, uh, everybody will lose
their jobs in 20 years. You know, this is something that
was said in the sixties. And there was a whole boom
around AI. The idea was to build those sort
of artificial worlds, right? That you could have a world
where you simulated some physical things in a computer,
and people thought that would have an impact, but it didn't,
right, in the business world. So after that, there was what's
known as an AI winter, which means a time where there was a
very little money in AI and very, uh, little enthusiasm
about AI. Then in the eighties, there was
another boom, right? And not millions invested.
And this was about expert systems, which is when you try
to encode all the knowledge of a person as rules inside a
computer. It didn't work well.
It was impractical to do. And there was another collapse,
if you want, of the AI field. Another bust of the bubble.
It also became almost shameful to say that you worked in the AI
field at that point, right? People said, stop saying AI.
They started saying, I work in cognitive something, 'cause they
didn't really want to say AI 'cause it was stigmatized.
Then we had after that period of, let's say, kind of calmness,
there was a much more moderate, if you want, boom of AI, uh,
that started to happen in the 2000s.
In 2010, it intensified, right? And that's the one that kind of
I lived through. There was a difference here
though, which is that AI in the form of machine learning.
So this was the machine learning boom, right?
The computer learns how to do stuff from data.
And the difference is that this time AI is successfully used in
commercial applications for the first time.
You have your Netflix recommendations and there's AI
there because the computer's learning from your behavior and
also what other users that are similar to you like, and then
you get personalized recommendations.
AI will sort search results when you're shopping for a holiday,
when you're searching for things on Google.
So AI has been with us for a while, and it's been used in a
commercially successful way, right?
And that's when I thought people were exaggerating about the
power of AI already. ' Cause they were saying we need to use
it for absolutely everything, right?
And then ChatGPT happened. That's kind of the a new wave,
yeah. Yeah, I thought there was gonna
be a sort of AI winter again before ChatGPT.
And there were signs of it. There were some industries that
were struggling, especially the self-driving car industry, for
example, that was doing really, really badly and still is.
But then with ChatGPT, everyone forgot about that, you know?
And it was like the most impressive boom that we could
ever imagine, yeah. Yeah, so I think, I feel that,
you know, all these hype about ML, I experienced it maybe back
then 20- , I dunno, 2010, 2015, those kind of era where you
start having like, I dunno, vision, OCR, those kind of
things. Recommendation system also
starts to appear. And then maybe a few cloud
computing also productize their AI services, right?
So I think that's when some of the craze about AI, you know,
people are trying to use that. Especially also a lot of
startups, you know, cool disruptors, so to speak, right?
They wanna also implement AI use cases.
And I think, yeah, this time is really so much different.
You know, the ChatGPT, the generative AI era.
So what do you think makes it such a craze now?
Because, uh, you know, you have studied the history before.
Um, but this time it seems so real and so different and people
are even talking about AGI and all that stuff.
I think there are a number of things.
So the first one is that this new generation of AI is used by
the general public directly. Just go into ChatGPT and use it.
The previous generation wasn't like that.
You know, as I said, there's AI recommending you movies on
Netflix, but you don't get to see any of that.
You don't use it yourself. But now with ChatGPT, you are
using it directly. So people have access to AI in a
much more straightforward way. But also, it's conversational
AI. And that's a big difference too.
And there's so much fascination around having a conversation
with a machine. And it's also one of the most
difficult tasks historically. The whole, the Turing test is
that. It's like can you build an AI
that is so powerful that you can't tell whether it's a human
or not? And this test is a conversation
where you chat with an AI, right?
So conversation is fascinating and it has so many applications,
and I think that's also a crucial difference with previous
generation of AI. Yeah.
And I find the barrier for you to use it is so much easier now,
right? So you can just go to a website
or even embed it in your phone these days, right?
And even you can speak to it the assistant way, uh, just like
what you mentioned, right? I think, yeah, so many people
just try, and they believe it's pretty smart.
Your book is titled Smart But It's Dumb.
So why do you call it? So I guess that's the first
thing about AI, right? Because people think it's smart,
but actually, maybe there's another side to it that we don't
know. I think that that was pretty
much the thesis of my book, which is that AI works really
nicely and it's impressive, until it's not.
Because you discover... I called it at that time an epic
mistake. Now it's called a hallucination,
right? People, that's the name that
people have adopted for it. But I've been warning people
about this for a very long time, which is that everything's nice
with AI until it makes a mistake that a human would never make.
Ever. And this reveals that it wasn't
actually smart, right? And those things again are,
they're very difficult to quantify and to even, you know,
define. But intuitively, all of a sudden
you're like, okay, this is really, really dumb.
And I've been observing this for a long time and I thought this
was, for example, the thing that would kill the self-driving car
industry. And it is the thing that did
kill the, it's almost dead. Like, uh, Cruise cars, you know.
The brand Cruise that builds self-driving cars, they have
more people in a control room than self-driving cars.
And they operate them remotely. They don't really operate, they
react to problems, right? And they happen so often they
need more people than cars, right?
The other company, Waymo, they also have a control room and
they intervene every three to five miles of driving manually,
right? And I knew this was gonna
happen. I've been saying it for years.
And the problem is that machine learning learns by repetition,
learns by example, right? So you need to show examples of
situations and then the computer kind of interpolates.
If it seemed to similar examples and there's something in
between, it can make deductions, but it's not very good at
processing situations that are far from anything the computer
has had access to the data. And when you're driving, there's
so many things that can happen that are completely out of the
ordinary. And for some reason though, we
don't really understand, humans are very good at that.
We seem to have these internalized models of the world
that we use to process things, to analyze things that, you
know, you see something totally different you've never seen but
you know how to react to it. And with machine learning, it's
notoriously bad at that, right? And I said, this is going to be
bad for the self-driving car industry.
And that's what's been making the adoption of AI so hard now.
Even now, generative AI in the enterprise world, right?
Or even to code, to code in a serious way.
People say, oh, it does 80% of the job, but then I have to go
and debug the thing that AI created, and it's causing me
more trouble than if I'd done it myself, you know.
Not always, but there's a, and it's again, these
hallucinations. And that's why I thought smart
until it's dumb is good way to summarize this, right?
Kind of describes the whole situation in a very short
sentence. Yeah.
Yeah, I think almost everyone, including software engineers,
right? I mean, we are, I mean I
perceive us as someone who understands how a computer works
and maybe we know about logical thinking and things like that.
Even us are still kind of like impressed sometimes when you use
coding assistants. You know, how come it can
generate a code that seems to work?
I mean, it works. There's a vibe coding as well,
right, these days. Uh, it seems to work.
But yeah, there are times where we think it would work, but
sometimes it just fail epically maybe, uh, borrowing the word
that you mentioned. And many, many catastrophic
mistakes actually happening as well, like deleting production
database and things like that. So definitely there's something
that we, like I personally, even though I use AI a lot, I don't
understand the fundamentals. Like I don't understand how all
these LLM, generative AI actually works.
Maybe it's also a good time for us to understand a little bit,
uh, about LLM, right? Yeah, yeah.
Yeah, sure, so we need to... um, a language model, very simple
computer program that tries to predict the next word based on
the previous word. Very, very simple.
And if you wanted to produce an entire sentence, you just make
it eats its own output. So you generate one word or a
token, should be a piece of a word from a vocabulary, it
generates a piece of text, you add it to the prompt, then you
run it through the program again, right?
That's all it does. Anything else is a sort of
illusion built on top. So for example, when AI seems to
search the web, there's actually another program, a wrapper
around the LLM, which intercepts an instruction that says please
go search the web. And then you go and you search
the web with another computer program.
And then you gather some results and you put them in your prompt
and then you continue. The user doesn't see this,
right? But under the hood it's just a
system to predict the next word. And it was trained on two types
of data. The first one is a huge amount
of data collected from the internet.
So if you want, this learns how to guess the best next word,
according to what internet have said.
There are lots of problems have been solved by people.
The solutions are online. And you can see that this can
take you pretty far but not completely far.
The system will not gonna learn how to do math, for example,
'cause people are not saying online two plus four equals six
with all possible numbers for the AI to actually get it.
So what AI is gonna do is going to create a plausible output.
Something that looks like a solution but not an actual
solution 'cause there's no data to support it.
Same thing with the names of citations of legal cases.
There's this lawyer uses ChatGPT to try to find legal precedence
for a case. And it kind of invents names
that sound good, but they're fake.
And I've seen that a lot of times.
So the second stage in training these AI models was to use
manual data created by humans. So what they did was they asked
people to generate lots of responses, and they ranked them
manually. They said, this is a good
response, this is a bad response.
They did a lot of, you know, trying to make it more
appropriate, not say inappropriate things this way
with people, you know. Paid very little money.
Some people say they hired people in Kenya for $2 an hour
to do a lot of this manual work. And then all this data was fed
back into the machine. This is called reinforcement
learning with human feedback to improve the models, make them
more aligned, made them better at solving well-known problems,
right? And that's how they're trying to
fix the hallucinations and all these problems.
But there's always like a long tail of problems that never
disappears. But yeah, that's the essence of
how... And another thing I want to say,
why, why now? Why, why does... because the way
this computer program works is different from what people were
doing before when they created language models which already
existed. In the past, they were using
something called an LSTM which is a very naive type of
technology, I think. I was always a bit skeptical of
it. 'Cause what it does is processes
one word at a time, and it tries to kind of summarize the entire
context up to now in one little vector of numbers that tries to
express what's been said so far, which is a huge compression of
all this information into a tiny little thing.
And then you use that to predict the next word.
But you've pretty much, you've forgotten a lot of stuff.
It's very, very hard. So what they invented was this
so-called transformer architecture that takes the
whole context and tries to represent the meaning of every
token in the context simultaneously.
They try to disambiguate the meaning of tokens by kind of
cross-referencing them, and then it predicts the next word from a
really, really rich representation of the context.
And this is what changed from previous technologies.
And this is something we've seen a lot in AI, which is that they,
somebody invents a new approach and that's when performance is
unleashed. It happened with convolutional
networks for image analysis, right?
And now transformer architecture for text generation.
Yeah, so it kind of is in a nutshell how AI works.
Very simple principles, very powerful, but also not all
powerful. Right.
And when people refer to generative AI, is it referring
to the same thing as LLM or is it something slightly different?
Well, that's complicated, first of all, because in academic
circles, generative AI does not mean what people usually mean by
generative AI. So forgetting about the
technical statistical definition, just generative AI
means AI to generate things. So it could be LLMs when we're
talking about language, but it could be also images, right?
And that's not an LLM necessarily, although you may be
typing in something or, you know, when you're transferring
this style from an image to another, you could call the
generative AI, but there's no language.
When you're typing in, there is a language model somewhere in
between, but you're generating an image.
But what I would say is it's mostly about generating new
content based on some cues or some prompts that we give to the
machine. Thank you for the clarification,
right? So every time I read about the
basics of LLM, right, the primitive understanding is that,
yeah, it tries to predict something based on a given
input, right? And you just multiply it with so
many inputs and maybe you train, you kind of like do
reinforcement learning and all that.
But somehow it just works magically, you know.
If you use some kind of assistant or maybe ChatGPT, they
will say, thinking, right? Thinking and then gives you some
explanation of what it tries to do, which if you read about it
seems very natural, like a human thinking and trying to deduce
stuff. And even we use it for coding.
Like I'm still amazed kind of like how come it can produce
code in a certain framework, in a certain language, that mostly,
mostly, uh, works syntactically correct and with a good output
and things like that. Like maybe explain us a little
bit. Like I'm still kind of like
don't understand in between how come it can be so powerful and
looks really smart. I think that internally, those
computer programs, right, the LLMs, they have states.
They're called hidden states. We can think of a hidden state
as a sort of abstraction that we don't necessarily understand
what it is. We let the computer learn its
own abstractions, right? So for example, the LLM may
learn that to produce or to make good predictions about code.
'Cause that's what it's trying to do.
It needs to have some sort of internal state that tries to
produce syntactically correct code.
As in it will maybe count the number of opening parentheses,
you know, so that it matches the kind of the closing parentheses.
So that's what made it, makes it really powerful.
There are many layers stacked on top of each other that have
hidden states that try to, when the model is trained, these
hidden states configure themselves if you want, in the
best possible way to solve the problem.
And because there are so many layers of them, we can't really
understand where they are, if we look because it's gigantic,
right? But I would say, because some
people, they become a little too critical of AI and they said
it's stochastic parrot. It just imitates or copy paste
stuff from the data, the training data.
Not really, right? What it does is it uses the
training data to try to configure those internal hidden
states. Initially, uh, I remember asking
ChatGPT, does an Anaconda fit in a shopping mall, right?
And it told me that it didn't because it's such a large
animal, right? What this shows you that there
was no hidden state that properly represented the size of
things or what things are big or what big means, right?
It was still producing good text in the sense that it sounded
correct, because its hidden states had properly represented
the grammar of English, right? But not necessarily the sizes of
things. But then this became quite viral
and as they improved the algorithms, they probably
generated manual data. I have to believe that they did
that probably somewhere where people asking questions about
the sizes of things and manually ranking the answers and that was
fed back to the LLM. And at some point, the LLM must
have created an internal hidden state that tries to represent
the things that are big in general, right?
And when things fit into other things, stuff like that.
We can't really know what it is 'cause it could use shortcuts.
There was a famous case of this image recognition algorithm that
or model that failed to detect a cow on the beach.
And the reason was that a cow was not on grass.
So the algorithm had actually learned that a cow is this thing
that looks like a cow and the grass underneath, because all of
the images and the training data had grass, right?
So it didn't really learn what a cow was.
So they can, and that this also causes hallucinations, right?
The AI may not really learn what we want from it.
However it does have hidden states that try to represent
things in an, with a higher level of abstraction, I would
say. And because they're stacked on
top of each other, you can combine them, right?
So there's a first layer where it tries to represent some low
level things, right? Is this a noun?
Is this a verb, right? At the next level, it could
represent, is this a noun that represents a big object or a
small object, right? And after lots and lots of
layers, you get a very, very good representation of the
meaning of things. And I think, yeah, the key to
the power of this is all this hidden stuff that goes inside
the machine. So I understand like sometimes
machine learning, you can't really explain what is happening
because it goes through, I don't know, like thousands or maybe
millions iterations, and that's why probably these hidden states
are kind of like embedded inside the model, so to speak, right?
And maybe now it's also a good time to understand about what is
called a model. Like people are calling
foundational models, you know, OpenAI, Gemini, Claude,
whatever. I dunno how many foundational
models are there. So maybe if you can explain what
is foundational model? Okay, let's start with model.
A model is essentially a computer program that tries to
predict something. So it's the essence of this AI.
There are different reasons for the use of the word model, we
could say it's trying to model the world, but I'm a bit more
practical with that. I don't like to use the word
program, 'cause there are two programs in AI.
One is the program that trains the other program.
You have a program, right, which is this model which an AI
engineer will configure, but they leave holes in it.
Things that are undefined that are called parameters, which are
numbers, right? So you create a sort of template
of the computer program, and then you need to run your
training, which is how to scan all this data to try to fill in
those blanks in the template. And that's another program.
And if we use the word program, we can, we get confused.
So which one is which, right? So typically you'll use the word
model to represent the final results of your training, which
is this program with all the blanks filled in, right?
And then you use the word training algorithm or something
like that to explain the alg-, the program that actually does
the training, right? So have a computer program
building another computer program.
The last one is the model. And I think with foundational
models, people mean very general purpose LLMs that can do lots of
things, right? And they can be used as building
blocks for other AI. So let's say you want to build
some, an app that is based on AI.
You may connect using an API to a foundational model, right?
Or you may take the foundational model that is open source and
fine tune it to your own data so it works better on your specific
problem, right? The foundational model would be
very, very generic. And then you have your specific
model to do what you want. Right.
So I think I'm always very fascinated by people who are
producing this foundational model.
I can't really imagine how they train it because it's such a
general purpose. You can use it for language, you
can use it for translation, math, whatever people wanna try,
right? I think it seems to be quite
smart and powerful. So these days, there are new
techniques that people are trying out, you know, things
like RAG and also agentic AI. So maybe if you can elaborate
these also shortly and maybe what other things that are new
that probably we are not familiar with.
Yeah. RAG means retrieval augmented
generation. It sounds very fancy, but it's
not. It's essentially a following.
You have, let's say in your company, you have a database
with a lot of your own private data, right?
And you want to search, using, use AI to search or analyze that
data. So what you do when the user
queries something, they search for something, or they ask in a
prompt for something. You run a separate algorithm to
go and find relevant documents that could help.
And then you insert the text of those documents in the prompt.
The user doesn't see this, but you say, okay, these are the
documents. So you need a large context
window. You need, uh, a model that
allows a lot of or very long prompts.
You insert all this data and then you add instructions.
You say, you know, answer the original question referring to
this data. What this lets you do is not
change the foundational model, not fine tune it to your data,
right? So it's a way to specialize an
AI to your own company data or any specific data without
changing the model. So yeah, that's RAG.
And the challenge is, okay, how do we find that data?
How do we find the relevant stuff?
But a good application of that is what we see with Google AI
results, which tend to be terrible by the way.
But anyway, we get to see AI using websites and analyzing
websites. And the way it probably works
under the hood is that first Google finds relevant websites
using its old algorithms. And then it takes the text in
those websites, puts it inside some prompt somewhere, it says
answer this query using also this information, right?
And then the LLM will generate answers which are then
intercepted and shown to the user.
So that's RAG. And then agentic AI or AI
agents, I feel like there's a new word every day, um, coming
up. And now agentic seems to be the
new word. An AI agent means that there's
an AI model that you use in a pretty autonomous way and you
execute some actions that come out of the model probably in an
automated way. So for example, you create a
little program that uses an LLM to generate some instructions.
And then you use that instruction.
I can imagine, I don't know, imagine an agent that scans your
emails of a corporate person who travels a lot for business and
you tell the LLM whenever you detect that this person wants to
travel somewhere, output this. ' Cause the agent can't really do
it. It's like output an instruction
that says search for flight for these dates in this specific
format. Then you connect that to an API
that goes and searches for the flight, for example, right?
And then you get the results and give them to the LLM and you
say, you know, send an email to this person with a proposal of
this could be your flight, right?
So essentially automate workflows and maybe even
connecting multiple AI agents. But essentially every agent is
nothing but an LLM to which you send a fancy prompt.
Right. So I think, um, when reading
your book as well, right? Explaining all these, uh,
fundamentals thing about the AI, LLM, things that we understand
now as like a cool thing, right? So definitely you can understand
some holes that could probably, uh, poke, you poke, right?
And you can start seeing, okay, why sometimes it hallucinates or
it makes mistakes, right? Simply because of the way it is,
I dunno, the fundamentals of how it works, right?
And maybe we can find clever ways of doing things like the
RAG and maybe the wrapper that you mentioned, right?
Don't forget. I think the wrapper is also
very, very powerful, I see. Because, um, you know, you can
tweak the workflow, so to speak, within the way AI replies the
response to you, right? I think the wrapper is kind of
like the intelligence as well that people build on top of the
foundational models. So I think thanks for
highlighting all these fundamentals.
So when people talk about AI these days, people think about,
I need to use AI. Businesses also scrambling so
hard to implement, adopt AI. First maybe adopt like ChatGPT
and all that within their companies.
And also building something smart, you know, using AI.
And I know in your book, you mentioned, you know, there are
some failure modes that could happen.
So maybe tell us how can business or organizations, you
know, start thinking about adopting AI or using AI within
their companies? Yeah.
Well, there are a few things. The first one is that I don't
think the best way to do anything is to have the hammer
and search for nails. You know, saying, what can we do
with AI? I've met companies that created
a whole team to promote the use of AI in the organization, and
it didn't go well because they were very biased.
They were, for the survival of the team, they had to say, this
can be done with AI, right? And they didn't acknowledge the
limitations of AI. In my experience, most AI or
most failed AI projects start with that saying, oh, what can
we do with AI? So I wouldn't do it that way.
Uh, it's good however, to be aware of what AI can do, what
the limitations are, because it's just another tool in the
toolbox, right? So when you find a problem,
something that could be done in a more efficient way in an
organization, you can see if AI will fix it.
I think the best way to use AI is to acknowledge these
hallucinations from the get go or just the limitations in
general of AI. Because if you embed that into
the, your product, you can build a successful product.
If you forget about that, you build a product that people
won't like. Because at one point, it would
be like, you told me this could do this job, but then it
hallucinated, you know, when I least expected it.
So one way to recognize the mistakes, for example, is to
have AI retrieve or point to specific paragraphs of, let's
say, documents and then show you the para-.
So it's so much different when AI interprets stuff than when
you use it to actually point you to something, for example,
right? Because it could help people
search for things more efficiently, right?
And if AI gets it wrong, then you still very quickly you can
see, you can say, okay, AI said this paragraph proves something,
but it actually doesn't, so I'll keep searching.
So I think we need to embed the idea that AI is not perfect in
the way we use it, and that's how you succeed with AI.
And another thing I think is, um, as a developer or in
general, in general we want to use AI when you can describe
your problem very succinctly. Because if you're doing
something so difficult, so custom that it takes you longer
to describe it in the prompt than doing the job, you don't
gain anything. So it needs to be something
that's very generic, very easy to describe, and then it needs
to be easy to validate, you know, the output.
A typical example is you, you know, you're coding and, uh, you
already know how to do this or you've done it before but you
forgot how to do it, right? So you go and ask ChatGPT, how
can I, you know, read a file from S3 using this Boto3 library
and then a very easy to describe.
And then ChatGPT outputs few lines of code which you've used
before but you'd forgotten how. And you can also very quickly
validate if it's working, so it's fine.
Some problems are very difficult to describe, right?
And, uh, and in that case, I think AI is not necessarily
going to save you time. And also sometimes the output is
very difficult to validate. Something you don't know at all
or a very complicated algorithm. And then you're like, you need
to just read someone else's code, someone else being AI.
And, uh, you know, you've read other people's code and
sometimes it's difficult to follow.
So you need to make sure, you know, to use AI in those easy to
describe, easy to validate scenarios, I think.
Yeah, I think that's, uh, pretty good tips, right?
Because I still think that companies try so hard to
implement AI for smart products, right?
So especially these days, like, I mean, almost every product
embeds something called AI features inside it.
It could be, you know, summarizing something and things
like that, right? But the most classical use case
is actually chatbot, right? The conversational thing.
Because it has been around even before LLM, right?
So I think a lot of chatbots. And now, uh, with LLM, I'm sure
people are still trying hard to create chatbots, customer
support thing. And we have seen many, many
cases where, you know, the agent responds something wrongly or
incorrectly, sometimes even give, you know, wrong result to
the customer such that it mislead them or the company,
right. So I think, definitely there's
this risk of hallucination. How can companies deal with this
hallucination? So is there something within
their solution or within their implementation that they can do
in order to reduce this hallucination from happening?
I mean, you can reduce hallucinations.
There are techniques. You can improve your prompts.
You can fine tune your model if it's not working well with your
own data. There's the whole thing called
chain of thought, which is I tell the AI first, build a
series of steps, a recipe to solve the problem and then solve
it, which some people say works better.
Uh, that's what they mean by reasoning models.
It's a whole charade. It's nothing really new there.
It's more like using this chain of thought thing.
But I would say you can't really eliminate hallucinations and
they happen when you least expect them.
So I would not use AI in the traditional way when
hallucinations matter. Hallucinations don't matter in
some context, right? If you're translating reviews of
hotels on TripAdvisor, nobody cares if they're a little bit
hallucinated than they are. Sometimes it's just the
translation's not very good. But I just wanna know if the
hotel is clean, if it's well located.
I don't need a perfect translation, right?
But you can't have it hallucination in literary
translation, right? And probably in customer service
chatbots, you can't have hallucinations either.
It happened to Air Canada that they, you know, it gave the
wrong policy to a customer and then Air Canada had to honor the
policy because a judge told it to.
It's like, you can't just tell the client in the chatbot, yes,
you can do this, we can get a refund for your flight, but then
no, actually you can't, right? So that's not a thing.
So if I were to work on this, I think I would use the chatbot as
a way to let people find the information they need
officially. So the chatbot can be a nicer
way, let's... you could try to interpret what people say, and
then point them to the right place which is the official
data. And you should be very careful
about not letting the chatbot, you know, interpret results
directly, you know. Dunno exactly how you can do
that, but I would frame it more in a way of how can I help you
find information, right? That kind of thing.
Another thing like I feel like a lot of people are praising AI so
much and the things that they can do, but it makes me wonder
if they just weren't doing things very well in the first
place. And it happens a lot with
customer service. I used to work for a company, it
was a travel company. And I understood the issue with
customer service, because... I never contact, you know,
customer service unless something really odd happened.
Otherwise I just go and try to find the information that I need
without having to contact anyone.
But that's not how everyone does it.
In this travel agency, people, they literally call to say, how
much luggage can I bring with me on the plane?
And this bombarded the customer service people.
And the first thing you imagine is, I'm gonna create a chatbot
for this. But I was wondering maybe the
problem they had was communication.
Maybe a few days before the trip, they should send an email
to people telling them, hey, you know, your flight is in two
days. This is the how much luggage you
can bring. And maybe that will solve the
issue, right? And I think there's a bit of,
you know, thinking AI can fix problems that you maybe could
have fixed. Another thing that doesn't cease
to amaze me is this whole thing of AI is so great for
programming because it eliminates all this boilerplate
code. And my question is, why do you
have any boilerplate code in the first place?
I don't have any in my work. I use libraries, frameworks to
avoid... Every time there's anything that
seems like will be a bit repetitive.
I already don't do it. I'm like, what, what library can
I use where someone has done this and or how can I avoid
this? I mean, some people I've said
this before, and some people told me, oh, but look at Django.
Like when you use Django, you need to, you create a new view
and you need to put the URL here, this there, and that
there. And I'm like, it's literally
three things. Is this what AI is really
helping you with? Like I still don't see why
people have so much boilerplate repetition, right?
So sometimes I wonder is there anything deeper here that maybe
you're not doing well and now AI lets you do it better, but maybe
there was a different way before, right?
Um, I, at one point, I asked ChatGPT for an algorithm to
calculate the position of the sun based on the time of the day
and your location. And it, on repeated executions,
it proposed different things. But one of them, it just
provided a very long algorithm filled with numbers and
trigonometry and it had constants in it like numbers.
And I was like, yeah, I could maybe copy paste this in my own
code. I will never know if this works,
if it makes sense. But there was a much easier
solution, which was like, pip install, sun position
calculator, which is the name of a library that does this, right.
Again, I had already, I'd already done this, right?
So I wasn't trying to get ChatGPT to do it for me.
I was more like, I was curious to see what would happen.
So I then I kind of executed the query again, and ChatGPT did
propose a library at another attempt instead of giving me the
code. But it kind of hallucinated the
way in which the library was used, how you called the
function. So when I tried to run it, it
was, it didn't run, right? I was like all this effort, when
in reality what you can do very, you know, every person who's
coded for a while knows that there must be a Python library
that calculates the sun position.
This must exist 'cause the ecosystem is huge and people
have validated this. It's a library that you can go
and check the community. Are there lots of comments in
the, and on the GitHub, you know, a page.
And there were a few actually libraries that did the sun
position, but very quickly it was okay, this one has a very
nice interface to use and there are lots of, it's an active
community. So, you know, in five minutes
it's running, right? So I don't mean to say that AI
can't help you because it can. But I also see a lot of that,
like are we forgetting about good engineering principles?
Are we forgetting about good customer service?
Can we understand what the most common queries are?
Created a frequently asked question section.
Send emails to remind people of stuff.
Cause a lot of your problems can be fixed by targeting the
underlying causes of the issue, not by using AI.
Yeah, I was laughing when you explained that, because I think
it's such a quite insightful, right?
So because people think just by applying AI, AI can do smart
things that can, you know, solve all the problems, you know,
seemingly, right? But I think we know, like it's
probably not possible to get a hundred percent.
And I was even frustrated sometimes if I get response from
like seemingly an AI, right? Uh, that tries to even first
re-explain what I just express as my complaint or, you know,
problem. And then it gives like some
basic information that I should have known in the first place,
right? And then I have to wait again.
So I think it's always very frustrating.
But these days people, when they build AI products, they seem to
always rely on this thing called eval method or eval test, to
actually, you know, kind of like train your solutions such that
it doesn't deviate so much probabilistically.
So tell us about this approach. Is it something that is useful
and people have to rely on? Or is it something also that
feels, uh, you know, like impossible to actually come up
with 100% accurate solution? Um, I don't know exactly that
approach, what it is, but I think I've been saying to people
for a long time, you need to, whenever you're going to build
an AI-based product, you need to create your own benchmark to
measure the performance of AI at the task you're trying to solve.
There's a whole process to do that, that is a little bit more
complicated than it seems, because you do need a validation
set which is separate from a test set.
Your validation set you use, you use it to select among different
models, among different options to pick the best one.
That's a little bit biased 'cause you have selection bias,
right? You could pick something that
works really well only on that validation set.
And that's why you need the test set at the end where you perform
an additional check, right? And you never go back.
If you don't like the result of the test set, you need to
collect a new test set with new data.
A lot of people don't do this carefully enough and you know,
it's up to you to see how far you want to go to that process.
But I've been telling people you do need a benchmark to start
with. And then once you have your
benchmark, you can see if a RAG approach works better.
If a fine tuning approach works better, you can compare
different models. There's also something pretty
catastrophic that happened a few weeks ago, which is that when
OpenAI launched their GPT-5, is it five?
The latest one? Yeah.
They removed all the previous ones, right?
And a lot of people were relying on previous models.
So if you're forced into a new model, then you need the way of
benchmark, benchmarking to understand if the performance
has changed on the problem you're solving.
So yes, now this will tell you how good AI is for the job.
It may not be good enough or may be good enough, you know.
It will help you assess, you know, the hallucinations that...
I would also be careful with just counting errors, you know.
You also may want to qualify them because sometimes just by
counting them, you don't realize how bad the errors can be and
the impact they can have. You know, people have been
saying a surgeon that is 99%, has a 99% survival rate versus
an AI surgeon that has 99.5, which one would you pick and
stuff? Okay, but does the AI surgeon
cut the wrong leg sometimes, even if he doesn't do it very
often, you know. That's a kind of a question.
But yeah, I think it's a good approach to do the benchmarking
to, you know, try to understand, do hallucinations matter,
because they probably will still be there.
Do errors matter? So do we still need a human into
the loop, do we not? Lots of things like that, that I
would definitely, is there another solution, like I said
with customer service. Can we do something else?
What's the real problem? Another question is, is the
problem I'm trying to solve a conversational problem?
Is there language involved? Because I see people do this,
they're building a travel planner where you actually input
destinations from a box. You select them from a box, like
a it, not a box, like a dropdown, right?
Or a search box. And then what it does is suggest
an itinerary in a very structured format.
Under the hood, this company was trying to solve it by using an
LLM. So they would take all the
user's query, transform it into a natural language problem, give
it to the LLM, then the LLM returns something, you would
have to parse that result to convert it into.
And it was super slow, for starters, right?
It was really, really slow. And I told this company, nothing
of this has anything to do with language.
And the easier solution that they actually found was we're
just gonna use a Google Places API, which just gives you, you
know, 'cause it was an itinerary planning to visit a place,
right? Or a city.
So you can actually get up-to-date opening hours, which
ChatGPT can't do. So you just go and get them from
Google Places and you use a typical algorithm for like
pathfinding, you know, like Dijkstra's algorithm, which is
there are libraries for that. So they kind of and that was
really fast. And they solve a problem without
any LLM. So LLMs are best used when
actually for a chatbot, for instance, when it involves
language. But a lot of people just trying
to use them for everything. Yeah, I think that definitely,
it's kind of like people think it's a holy grail, right?
And in fact, I think sometimes maybe because of the hypes,
right? People just creating new
libraries, new tools, new stuff that works, that relies on LLM
or maybe prompts, so to speak, right?
They kind of like fit in, you know, the solution with these
prompt engineering kind of thing, right?
Try to come up with a, you know, as best prompt as possible, give
as much context as possible. But you also mentioned, you
highlight when the foundational model change.
Definitely, things probabilistically will change,
maybe sometimes by a lot, maybe sometimes not by a lot.
But still you need to kind of like validate and test your
approach, right? Because when this happens,
probabilistically, you might give a wrong output and
suggestion as well. So I wanna move on to the next
section by, you know, discussing about the concerns people have.
Uh, one thing about AI, it creates a lot of possibility,
but there are also a lot of concerns from people that, you
know, what will happen to my job, what will happen to my
role, what will happen to, you know, the world economy and all
that. So maybe from the first
discussion, I wanna highlight the concerns, right?
Uh, especially for tech, software engineers think there
are many, many software engineers that can be replaced,
including juniors. What is your opinion about this?
Yeah. Well, I-I've written a lot about
this in my second book on AI, which is called the AI Pocket
Book, which we haven't spoken about yet.
But part of the goal of that book was to answer these
questions. I was approached by a publisher.
They thought, we need a book on surviving AI for software
developers, right? And I was like, okay, I will
definitely speak about the whole job market situation.
I think that there will be jobs that will be lost to AI for
sure, maybe prematurely sometimes.
'Cause we've already seen companies firing people and now
hiring them back because it didn't work.
So I think that, I mean, this is, this could be a, you know,
we can speak about this from many angles, but for the angle
of a person who wants to preserve their job, I would say
that there are different kinds of jobs.
Some jobs, AI may be able to replace, and some, it won't.
So as a professional, you want to position yourself in that
latter category, right? I think that there's a certain
level of work that requires a lot of excellence.
When you go above and beyond, when you rely on a network of
other people, you need interactions, you need multiple
skills. In the business world, that job
is oft-... In the software world, it's
often at the intersection between software and business.
So you are not a person who takes a specification of a UI
and codes that specification to the letter without an opinion,
right? You are the person who will help
the business choose the best feature to build.
There are lots of positions that are like that.
If you work for startups, for example, you may need to be a
jack of all trades, a person helping select features.
As you progress in your career, you're closer to the business
very often. So my advice is often try to
work at the intersection between business and technology.
See yourself as a person who helps build commercially
successful products. Not just write software.
Because I think you will be quite irreplaceable when you're
that person. And another thing I think is
that a lot of people speak about other people's jobs.
They say these people will lose their job, but they've never met
a person who does that job and they don't know what it takes to
do the job, right? Especially at the excellent
level, right? I've seen this a lot.
You know, people say screenwriters will get replaced
with AI or replaced by AI. But then, have they met a
screenwriter? Have they seen what it actually
takes to produce a show that people love, that people laugh
at or laugh with, you know? I am listening to this podcast
called Office Ladies, which is, by the actresses from The Office
where they tell you stories behind the show and you can see
how much thinking there was sometimes about certain
storylines, you know, that they decided to include in the show.
There's so much that goes on, you know, in writing a show.
Yes, there are some cookie cutter show, soap opera kind of
show that maybe AI can do, but there's so much more that
happens when people are writing a show.
And the same thing with programming with code.
I feel like maybe I'm also a bit more senior in my career, but
most of my work in software is not writing code in terms of,
you know, I'm gonna move this button from here to there.
There's a lot of thinking about security, about architecture.
A lot of this has to do with negotiating with the business.
A lot of it, you know, the business wants to do this, but
this is really big and they don't know it.
So I explain to them why it's big and then I propose a
different solution. What if we do this other thing?
It's not as great, but we can do it quickly and then we can see
how people react. And then we see if we build the
bigger thing that you want. All of these help, you know, as
part of software development, but it's not writing lines of
code. So if I had to recommend to a
software developer, again, I would say, a lot of interaction
with business, a lot of high level work business, learn about
business, you know, read business books.
'Cause a lot of us, we have no idea about anything related to
business, to sales, to value propositions.
How to define a value proposition, for example.
There's a book called Value Proposition Design, which I,
which I recommend. So yeah.
I think that a good way to protect that your job is that,
you know, yeah. Yeah.
So I think that's very good advice, right?
Because obviously when we talk about AI replacing jobs is
probably the first, will be the mundane things.
You know, like producing lines of code, translating from, you
know, one thing to another, or even like maybe when you say
about screenwriter, maybe not necessarily screenwriter, but
someone who polish, you know, some writings that, some drafts
that people do, right? So those kind of things,
definitely the first prospect that could be replaced, right?
But if we keep moving to the higher level.
Like when we talk about programming language, also the
same, right, it started with like machine language then
becomes general programming language.
Now you have conversational language.
If we can think of ourselves moving to the higher abstraction
maybe, so to speak, maybe we can even amplify, you know, the
value that we can produce, right?
So I think I agree what you said that software engineers need to
understand more about business or maybe thinking in terms of,
you know, how can we make this more successful, commercial,
some psychological aspect, human aspect that probably software
engineers can also explore. So how about the juniors here?
Because I think the juniors might be kind of like
frustrated, right? So I just finished my study for
example. But now I'm finding it hard to
find a job. So do you have any advice for
them? I mean I spoke about this
excellence side of things, but there are other things that will
be helpful. One of them is when you work on
very custom, customized work if you want, that is very kind of
complicated and narrowly specified and stuff.
If you work on complicated stuff, and I can give you a
couple examples, those things are unlikely to be replaced by
AI, because first of all, you can't even describe to an AI
what you need to do. Even the business people will
not understand. And, uh, I can give you an
example. I worked on a project that
involved modeling the propagation of temperature or of
heat inside a building, right? And I was writing the software
for this, right? All these complicated
calculations, but it was so custom and they were devices
connected to buildings using APIs where we get the data and
then physical models. And there were parameters that
define, you know, the insulation of the building.
It's very hard to, for me to even imagine how you could
describe all of that in a prompt which is not the same as writing
the code essentially, right? Uh, so I would say you want to
specialize or to gain experience on tightly controlled kind of
software. An example of what's not like
that is, you know, building an prototype just to show something
to a potential client, but it doesn't really work.
There's a lot of MVP or no-code kind of thing going on there.
It's like we're gonna build not a really functional thing that
works with lots of clients, with lots of customer requirements,
we're just gonna build a very rough prototype to validate an
idea, right? I think those are the jobs where
people are gonna be replacing that with AI a lot.
But then you can do very, very custom kind of like, focus on
really tightly defined, custom complicated problems.
You know, if they can be a bit scientific or they require
mathematics, you know, it's likely business people will not
have studied those things either.
So they won't be able to know where to start with those
problems. Also working on anything that is
mission critical will help you. When I say mission critical,
something that you deploy a piece of software that could
lose a company a lot of money if it wasn't used correctly or it
could even cause harm to people. Because this is going to require
a lot of manual validation of manual input of, you know, I
don't think that's going to be replaced by AI anytime soon.
So if you can become the person who learns about those things,
then you'll be better placed to find a job or to preserve your
job. So I hope people could learn
from these tips. So when you mentioned about
building prototypes, right? My mind straight, straightly go
to vibe coding, right? Uh, I think many people think,
you know, now everyone can code, you know, just doing vibe coding
and even it might be creating a perception that you can be a
vibe coder, you know. There might be a job role as a
vibe coder specializing in vibe coder.
Same thing like prompt engineering.
You know, I can be a prompt engineer that can solve, you
know, maybe general purpose AI conversational kind of a thing.
So maybe tell us, uh, your view about this and, and will in the
future, we see new job types being created, something like
this, or is it more like a hype now?
Now I think there is a place for everything and there is a place
for a job title that is a vibe coder potentially, because we've
seen that with Bubble, you know, Bubble, the no-code tool.
I've seen people who specialize because you need to actually
learn how to use those tools at the end of the day, especially
if you want to build something custom.
Which is kind of part of the problem here, is if you just
type it a little prompt and you accept what comes out of it,
then it's fine. But if you want to start, you
know, like with no-code, you often need to learn what a
platform does and how to use it, how to connect it to data
sources and all of that. And I've seen people who
specialize in helping companies build with Bubble, right?
Because it's cheaper, it's quicker, right?
You're not gonna build production-grade software.
That's the illusion that we need to kind of eliminate.
And whenever you want something really custom, it's as difficult
as coding. Because coding is that.
It's literally telling the machine what you want it to do,
right? So the illusion that, that's the
same as building production-grade serious
software, I think that illusion needs to go.
But there's definitely room for that other thing in between
where you help with MVPs, where you help with prototypes, right?
We could see UI designers maybe using AI to create better
prototypes for their clients, which are more functional than
before. So there will be changes
definitely in there. And there will be new jobs and
new opportunities, I think. Yeah.
Yeah, so hopefully as with a new, I dunno, technological
disruption, I think there will be a lot of fears in the very
beginning. But hopefully after that we can
like thrive and, you know, maybe in the new opportunities people
move on to new kind of skillset and solving bigger problems and
things like that. So hopefully that will be the
case. And speaking about that, right?
So I wanna touch on this one thing that I think it's, uh,
worth to discuss shortly, right? So the AGI possibility.
So I think many AI companies think AGI will happen, I dunno,
three years, five years, whatever that is.
Sometimes I listen to podcasts as well, you know.
There are seemingly AI experts telling that humanity is in
crisis. So personally do you think AGI
will happen or maybe some resemblance of it?
So yeah, maybe a little bit on this part.
Look, the first thing, okay, AGI, artificial general
intelligence, right? A computer that can do anything
a human can do. It's not very well defined, but
we're talking about really super powerful AI.
Well, the first thing I want to say, it sells so much to make
those predictions. There's nothing that sells more.
People have built entire careers around predicting that.
You know, uh, an example is Ray Kurzweil who's written these
books on the singularity, saying the singularity is going to
happen. Singularity is when the machines
become so smart that they can improve themselves at an
exponential rate and everything collapses.
And it's like like sci-fi world. It's, people are fascinated by
that and it sells so much. I feel like people who speak the
way I speak about AI, and I've heard the same experience, you
know, from other people, we actually struggle more to be
heard because people don't invite you to give talks as
much. Who do you want on the stage?
There's a reasonable person telling you AI is smart, but
it's also kind of dumb. Or do you want a person telling
you how it's going to be the Terminator, you know?
And it seems like the latter is, it sells a lot.
And when I was looking for a publisher for my first book on
AI, I remember I spoke with an agent that he's a pretty
prominent agent. Like a book agent.
And he told me, no, like your book won't sell because you have
to either say that AI will be so amazing, that will change
everything, or they will destroy everything.
If you say something kind of in the middle, nobody really like
that. So the first thing is like this
AGI thing is, and it's also self-serving because a lot of
people are making money from selling the shovels or selling
the AI to build AI or AI for the sake-, and then they tell you
AGI is coming, right? I would tell you the answer is
no. So AGI is not coming anytime
soon. And the reason is that the
current methodology used to build AI, which is machine
learning, has some flaws. And these flaws cause the
hallucinations, for example, which will prevent AGI, right?
They are not solvable right now within the methodology that we
know. This means that if a new
methodology, something different from the current type of machine
learning were invented, maybe it could be solved.
I don't know. There's also a question that in
principle, this is possible or not.
But even if we put that aside, if a new methodology is
discovered, maybe. But nobody knows that
methodology now, right? We don't know it and we need to
acknowledge that we don't know it.
When GPT-5 came out - and it was a flop, a big flop, and a big
disaster - cause it was nothing new.
It was maybe a little bit better, but not really.
I wasn't surprised because nobody has discovered that
secret ingredient or if you want that new methodology to solve
hallucinations and stuff. So to me, it was no surprise
that GPT-5 still hallucinated because they haven't discovered
anything that I know of in between, right?
And you can't predict scientific discoveries like that.
You know, the transformer architecture was the discovery
that led to LLMs or like to the current powerful LLMs. What's
the next discovery? We don't know.
We don't know when it's gonna happen.
I always say the example of nuclear fusion based power,
they've been telling us for decades that that's the future.
It's around the corner. The problem is that the
methodology hasn't been found, hasn't been discovered to do
that. And they can keep putting money
into it and maybe that will help, but we don't know if it's
gonna happen or when, right? And that's the situation with
AGI, right? I feel like a lot of the
philosophical questions around AGI, they're valid and it's good
to ask them. What would happen?
Like what happens legally if... Does a robot have feelings?
I like those questions. I think they're fascinating, but
they are not the reality of current machine learning.
Like this thing of I'm going to learn from data how to predict
the next word. It's not going to lead us there
and it's gonna keep hallucinating.
But then if a new thing is discovered, you know, maybe.
Right. Pretty exciting answer, right?
So I think people sometimes, uh, I don't know, sometimes yeah,
they project something that seemingly bombastic, and then
yeah, it sells like what you mentioned, right?
But we have seen it, with the ChatGPT-5 release, right?
So it kind of like flop a little bit.
So maybe, you know, the AI hype is not as great as people sold
it to be, right? But obviously these next few
years will be very, very exciting, unpredictable at
times, right? Because there will always be new
things being invented and we just have to adapt, I guess.
My stance now, even now, is just, yeah, see, adapt, use them
as much as possible, understand what it's capable of and not
capable of, which is also very, very important.
And try to fit in you, yourself, your skillset, your value, into
those landscape, right? So I think probably that's like
the most practical thing. So Emmanuel, thank you so much
for this conversation. So I know we are a little bit
over time, but I have one last question that I have to ask you.
This is like a tradition in my podcast.
I call this the three technical leadership wisdom.
So if you think of them just like advice, uh, would you be
able to share your version today?
Yeah. So I would say, look, the first
one is that there's a term called a moat.
As in M-O-A-T, right? Like a protective moat.
It's also called a competitive advantage.
I think a lot of people don't understand much what this means
or they've read some books that tell 'em this is a competitive
advantage when it's actually, it's actually not.
I think there's a lot of confusion around the competitive
advantage, and I've been doing a lot of research myself on the
matter. What I mean with all this is
that you need to understand that building a successful business
is a business problem. It's not a technology problem.
It's usually, it doesn't, no business has failed because you
use Python 3.5 instead of, you know, whatever.
And the role of a business person or even the goal of a
business is, even if we don't like it, is to create a sort of
monopoly. You want a moat that protects
your business from competitors. You need access to some, a
special way to keep your clients or to gain more clients or to
access resources cheaper than anyone else.
Like there needs to be a special thing.
And that thing is called a moat. And one of the reasons I tell
you, I think technology leaders need to keep this in mind is
that there are a few ideas going around that are terrible.
One of them is this 10x thing. You build technology that is 10x
better than other technology and then you're done, your
businesses. That's not true.
And the proof of that ChatGPT was definitely, I think a 10x
kind of thing. And the business is doing
terribly. They're losing billions every
year. And the problem is they have no
moat. There's nothing special.
Having a specific type of technology doesn't build a moat
around your business. What is aggravating here is that
everybody knows how ChatGPT works, right?
Because it wasn't even invented by OpenAI.
It was invented by Google researchers who published the
work on a scientific paper. And I've been saying this to
people, right, a lot. And I kind of predicted this, I
wrote a book called Siliconned also where I wrote about this,
about OpenAI and stuff. And then what happened now is
Deepseek comes out. And then the world is like, oh
my God, it works as good as, you know, OpenAI models and what's
this gonna mean? Yeah, we knew this was gonna
happen because there's so much money to be made probably in
the, in AI that a lot of other entrants, competitors, copycats
are going to try to build the same thing.
So you do not actually build a successful business around
having superior technology. You build them around, for
example, network effects that keep people engaged.
Or you build them around switching costs.
People don't want to leave you because they're kind of stuck,
you know. With Apple, you have all Apple
ecosystems, so you don't want to switch to a different kind of
phone. Those are the things that
actually make or break a business.
It's not the technology. Um, another thing that I think
is very prominent in the technology world is a search
cost. Not a switching cost, a search
cost. Meaning that you want to
capitalize on the fact that people will find it annoying or
costly to search for an alternative.
I think Canva benefits from that, right?
People don't want to use anything else because they know
Canva and they're not gonna start searching for Canva, and
they don't even have opportunities to find
alternatives. Whenever they want to do any
simple graphic design, they just go to canva.com.
But again, why I'm telling you this?
Because the key to the success of this business is that the
product is cheap, right? If Canva tries to charge a lot
of money, this search cost starts to not protect the
business anymore because people are gonna start shopping around.
They're gonna, oh, Canva is now 200 a month.
Can I use something else? And business strategies about
this. And I think a lot of
technologists like me, we had, I didn't learn any of this in
university. I learned how to code.
I learned how to create really fancy SQL statements.
But that doesn't make or break a business.
It's business side of things and the competitive advantage is
super, super important. So that would be my first, uh, I
dunno what you call it, my leadership something.
Second one, there's a lot of boring stuff to be done.
The most successful entrepreneurs I've met have
built something really boring to help a boring industry do
something that was super inefficient, right?
And speaking of competitive advantage, I think one of the,
or the moat that they had was that it was such a niche thing
that it's just, it's actually a small, small market and people
don't want to enter the market. Once you're there, your clients
have switching costs that are really, you know, high.
And I've seen this a lot. I've seen it in aviation a lot.
A lot of stuff is done on paper nowadays.
People literally, they fill in paper forms even today to track
the maintenance of an aircraft or stuff like that.
There's so much stuff that's done badly.
And I think there's a lot to do in the world and in that kind of
boring arena. But it's not flashy, like
building the new goggles to pilot your drone or whatever.
And you know, and it's like, but actually it's a lot of fun to
work on things that people actually use and care about.
You know, there was a person who recently quit a job at a very
fancy startup and joined another company that is, I can't name it
by name, but it's a company you've heard of and it's a
company that builds software that is actually very useful.
And I think it benefit from the sort of Canva effect that people
just go and use it because it's cheap.
They're not gonna shop for alternatives.
It's doing really well. And she was telling me that she
was actually, she was a bit concerned.
She was like, oh my God, I'm going from this alleged unicorn
which is never happening, you know.
The company has never actually turned it into a unicorn, the
stock options are worthless. And then she goes into this.
But it's actually been very exciting for her to work on a
product that people use, even if it's not sexy.
'Cause this is not a sexy product.
So don't forget about the boring stuff.
So that would be my second thing.
And then the third one, which I already said, but I think I
would like to go back to that, is that if you are building
anything based on AI, you should acknowledge the limitations of
AI from the very beginning and embed them in the product.
And that's how you're gonna build a successful product,
right? If AI hallucinates, maybe do
something so that, again, you quote a passage from the text,
you do not interpret it. You don't leave room for
hallucinations. Um, I've spoken to companies
that are doing a good job in this sense, right?
They are using, or they're building an AI-based product
that helps, that generally helps save time for people without
pretending that it can do the job in a flawless way and people
are liking the product, right? And I think that's something you
need to consider. Yeah.
So those are my three. Wow.
So I find it really insightful and very beautifully said as
well. So the boring stuff, I think I
like it the most, right? Because sometimes we are so much
into the hype, right? So we chase all these hypes.
But definitely there are things that are still working.
Maybe it's boring, but it's useful and people use, and it
creates value in the world. So I think thanks for reminding
us those things. So Emmanuel, if people like this
conversation, they wanna reach out to you, ask you more
questions or discuss about, you know, AI stuff.
Is there a place where they can find you online?
Best place is LinkedIn. I usually post stuff on
LinkedIn. I, yeah, that's where I'm,
that's where I hang out. I know it's a little bit weird.
Some people will tell me, you know, why are you on LinkedIn
and not on Twitter? But that's the main place for me
to communicate with people. Um, and, uh, you can also find
my books to learn more about the things I have to say.
And you can send me a message on LinkedIn.
Yeah. Right.
So I highly suggest people reading your book, especially if
you want to demystify the AI hypes that is happening now.
Like try to understand the power of it, the fundamentals of it,
and the pitfalls where AI might fail.
So thank you so much for writing those books Emmanuel, and for
this conversation. So I hope you enjoy this
conversation and thank you again.
Thank you.