1 00:00:00,000 --> 00:00:03,216 Everything's nice with AI until it makes a mistake that a human 2 00:00:03,216 --> 00:00:04,380 would never make. Ever. 3 00:00:04,380 --> 00:00:06,870 And this reveals that it wasn't actually smart. 4 00:00:06,900 --> 00:00:08,340 You can't really eliminate hallucinations. 5 00:00:08,390 --> 00:00:09,660 They happen when you least expect them. 6 00:00:09,810 --> 00:00:12,875 It happened to Air Canada. The chatbot gave the wrong 7 00:00:12,875 --> 00:00:16,258 policy to a customer. And then Air Canada had to honor 8 00:00:16,258 --> 00:00:18,450 the policy because the judge told it to. 9 00:00:18,540 --> 00:00:22,080 Emmanuel Maggiori is AI industry insider, who has developed AI 10 00:00:22,080 --> 00:00:25,408 for a variety of applications. He is the author of the books 11 00:00:25,408 --> 00:00:28,080 Smart Until It's Dumb, Siliconned, and The AI 12 00:00:28,080 --> 00:00:29,946 Pocketbook. In your book, you mentioned 13 00:00:29,946 --> 00:00:31,020 about two AI booms that happened. 14 00:00:31,080 --> 00:00:33,570 But actually it ended up going nowhere in the end. 15 00:00:33,600 --> 00:00:35,550 So maybe tell us a little bit of this history. 16 00:00:35,610 --> 00:00:37,590 There was an AI boom in the sixties. 17 00:00:37,590 --> 00:00:39,900 Everybody will lose their jobs in 20 years. 18 00:00:39,900 --> 00:00:42,830 That was said in the sixties. Then in the eighties, another 19 00:00:42,830 --> 00:00:44,798 boom, another collapse. There are concerns, what will 20 00:00:44,798 --> 00:00:46,745 happen to my job? Especially for tech, software 21 00:00:46,745 --> 00:00:49,500 engineers, many that can be replaced, including juniors. 22 00:00:49,530 --> 00:00:52,029 What is your opinion? There will be jobs that will be 23 00:00:52,029 --> 00:00:54,760 lost to AI, for sure. Maybe prematurely sometimes. 24 00:00:54,760 --> 00:00:57,129 We've already seen companies firing people and now hiring 25 00:00:57,129 --> 00:00:58,629 them back because it didn't work. 26 00:00:58,750 --> 00:01:01,239 Business is scrambling so hard to adopt AI. 27 00:01:01,300 --> 00:01:03,640 How can business start thinking about adopting AI? 28 00:01:03,670 --> 00:01:06,550 Building a successful business is a business problem. 29 00:01:06,550 --> 00:01:09,340 It's not a technology problem. I've seen so many projects fail 30 00:01:09,340 --> 00:01:13,530 almost the same way every time. The best way to use AI is to 31 00:01:13,530 --> 00:01:15,815 acknowledge these hallucinations from the get go. 32 00:01:15,845 --> 00:01:18,866 If you embed that into your product, you can build a 33 00:01:18,866 --> 00:01:20,629 successful product. Then they tell you a AGI is 34 00:01:20,629 --> 00:01:22,059 coming, right? I would tell you the answer is 35 00:01:22,059 --> 00:01:24,155 no. AGI is not coming anytime soon. 36 00:01:24,215 --> 00:01:43,086 And the reason is that... Hello, everyone. 37 00:01:43,086 --> 00:01:46,194 Welcome back to another new episode of the Tech Lead Journal 38 00:01:46,194 --> 00:01:48,996 podcast. Today, I have with me someone 39 00:01:48,996 --> 00:01:52,935 very excited, to, you know, share some learnings or some 40 00:01:52,935 --> 00:01:55,680 insights that he has been dealing with regarding AI. 41 00:01:56,160 --> 00:01:58,813 I know maybe this is like on everyone's mind these days, 42 00:01:58,813 --> 00:02:00,770 right? Emmanuel Maggiori actually wrote 43 00:02:00,770 --> 00:02:04,743 two books related to AI. The first one is called Smart 44 00:02:04,743 --> 00:02:07,908 Until It's Dumb. And the second book is titled 45 00:02:07,908 --> 00:02:11,133 The AI Pocket Book. I find this a rare opportunity 46 00:02:11,133 --> 00:02:15,028 for us to learn more about AI. What actually these days, you 47 00:02:15,028 --> 00:02:18,442 know, general- generative AI, LLM, all these tools that we 48 00:02:18,442 --> 00:02:20,691 have been hearing a lot and using a lot in our day-to-day 49 00:02:20,691 --> 00:02:24,021 work, try to understand the basics and maybe few interesting 50 00:02:24,021 --> 00:02:27,238 things that Emmanuel can share today for us to learn from him. 51 00:02:27,568 --> 00:02:29,548 So Emmanuel, thank you so much for this opportunity. 52 00:02:29,548 --> 00:02:31,834 Welcome to the show. Thank you for having me. 53 00:02:32,646 --> 00:02:33,656 Right. Emmanuel. 54 00:02:33,656 --> 00:02:36,216 Uh, I always love to invite my guest first to maybe share a 55 00:02:36,216 --> 00:02:38,416 little bit more about you by sharing any career turning 56 00:02:38,416 --> 00:02:40,540 points that you think we can learn from you. 57 00:02:41,686 --> 00:02:45,596 Career turning points? Yeah, I think there were two 58 00:02:45,596 --> 00:02:48,634 important ones. The first one was when I 59 00:02:48,634 --> 00:02:51,820 realized that a lot of the technology industry was driven 60 00:02:51,820 --> 00:02:55,234 by hype. You know, a lot of projects were 61 00:02:55,234 --> 00:02:59,374 just, I don't wanna say fake, but a little bit fake. 62 00:02:59,434 --> 00:03:03,970 Sometimes, not on purpose. But you find yourself working on 63 00:03:03,970 --> 00:03:08,701 products or on building products that don't make any sense and 64 00:03:08,701 --> 00:03:13,747 everybody knows it, right? There's a difference between an 65 00:03:13,747 --> 00:03:16,459 experiment that is worth doing, right? 66 00:03:16,459 --> 00:03:19,477 Okay, we don't know how the client will react to this, so 67 00:03:19,477 --> 00:03:22,369 we're still going to build it as a sort of MVP. 68 00:03:23,249 --> 00:03:25,589 There's a difference between, you know, that, and just 69 00:03:25,589 --> 00:03:27,829 building something that doesn't make any sense. 70 00:03:27,889 --> 00:03:31,951 Everybody knows it, but you're still building it because a VC 71 00:03:31,951 --> 00:03:35,411 told you that it was a good idea, or because an upper 72 00:03:35,411 --> 00:03:38,599 manager from a company just gave you a lot of money to do this 73 00:03:38,599 --> 00:03:41,593 and then you're just doing it. And unfortunately, I found 74 00:03:41,593 --> 00:03:44,945 myself involved in projects like that a little too much. 75 00:03:45,905 --> 00:03:49,815 And at one point, I thought it seems like most of it, 76 00:03:49,815 --> 00:03:53,490 especially when you work on cutting edge stuff, it's like 77 00:03:53,490 --> 00:03:56,706 that. And I think it was a turning 78 00:03:56,706 --> 00:03:58,805 point because I was very naive before, right? 79 00:03:58,805 --> 00:04:00,965 I thought companies are efficient. 80 00:04:01,115 --> 00:04:05,192 You know, why would a company ever carry out a project this 81 00:04:05,192 --> 00:04:07,560 long when it's obvious that it will never work. 82 00:04:08,260 --> 00:04:11,148 But you know, they do. Things like that can last for a 83 00:04:11,148 --> 00:04:14,048 very long time and these projects, um, they get swept 84 00:04:14,048 --> 00:04:17,642 under the rug very easily. So I think that was an important 85 00:04:17,642 --> 00:04:21,207 career turning point. And I think another one, which, 86 00:04:21,207 --> 00:04:24,751 you know, well, that first career turning point led me to 87 00:04:24,751 --> 00:04:27,485 the book Smart Until It's Dumb, what you mentioned, right? 88 00:04:27,485 --> 00:04:30,405 It was, I need to tell the world that AI is not that great. 89 00:04:30,970 --> 00:04:32,950 That there are lots of problems with it. 90 00:04:32,950 --> 00:04:35,794 That lots of projects fail. And why? 91 00:04:35,824 --> 00:04:37,680 So that was why I wrote that book, right? 92 00:04:37,740 --> 00:04:42,360 And then another turning point was, you know, I thought I'm 93 00:04:42,360 --> 00:04:47,177 gonna work on more foundational stuff, less hyped up stuff. 94 00:04:47,207 --> 00:04:50,207 Just do some more traditional software engineering. 95 00:04:50,687 --> 00:04:54,322 But then I realized that a lot of people weren't working much. 96 00:04:55,427 --> 00:04:57,717 They weren't doing anything, right, including me. 97 00:04:57,717 --> 00:05:00,437 I found it hard to find work to do. 98 00:05:01,084 --> 00:05:03,948 I was hired by a company and I thought it was going to be super 99 00:05:03,948 --> 00:05:07,628 exciting and they kind of put me in this staging area to wait 100 00:05:07,628 --> 00:05:09,798 until I was assigned to a project. 101 00:05:09,948 --> 00:05:13,904 And there were lots of us in there being paid to do nothing, 102 00:05:13,904 --> 00:05:16,896 right? And that, you know, I thought 103 00:05:16,896 --> 00:05:18,768 something's really strange with this field. 104 00:05:19,038 --> 00:05:21,966 It seems like there's a lot of money flowing into the field, 105 00:05:21,966 --> 00:05:25,056 lots of projects. They get launched without really 106 00:05:25,056 --> 00:05:29,066 knowing what, you know, they're gonna be or what we're gonna do. 107 00:05:29,066 --> 00:05:33,143 And companies like to hoard staff sometimes. 108 00:05:34,203 --> 00:05:37,544 And I thought, again, this is companies not being very 109 00:05:37,544 --> 00:05:39,202 efficient and I didn't think it would last. 110 00:05:39,308 --> 00:05:40,778 And then everybody got fired, right? 111 00:05:40,868 --> 00:05:43,928 Twitter fired 80% of its employees and everybody. 112 00:05:43,988 --> 00:05:48,431 You know, Meta, all major companies, made a lot of people 113 00:05:48,431 --> 00:05:49,964 redundant. And I thought, you know. 114 00:05:50,364 --> 00:05:52,751 I mean, some people work very hard, but a lot of people, they, 115 00:05:52,751 --> 00:05:55,134 they're just, they just don't know what to do. 116 00:05:55,134 --> 00:05:57,574 When I thought that wasn't sustainable. 117 00:05:57,574 --> 00:05:59,304 So that was another learning for me. 118 00:06:00,355 --> 00:06:01,849 Wow! Very interesting sharing that 119 00:06:01,849 --> 00:06:05,713 you just shared just now, right? So I find, I mean, when you said 120 00:06:05,713 --> 00:06:09,203 that I also had experience, uh, of those kind of, you know, 121 00:06:09,203 --> 00:06:12,242 projects, career, right, where, you know, seems things are going 122 00:06:12,242 --> 00:06:14,592 to the wrong direction, but we still march on anyway. 123 00:06:15,132 --> 00:06:17,722 We think that something positive will come out of it. 124 00:06:18,052 --> 00:06:21,107 And the second one, you know, especially for consulting or 125 00:06:21,107 --> 00:06:23,266 professional service kind of work, sometimes, yeah, we were 126 00:06:23,266 --> 00:06:26,242 put on the bench or on the beach, some people also call it. 127 00:06:26,675 --> 00:06:29,937 And maybe in some big organizations, there are roles 128 00:06:29,937 --> 00:06:33,769 which have maybe little things to do and, you know, spend their 129 00:06:33,769 --> 00:06:37,163 time doing, you know, something else which is not necessarily 130 00:06:37,163 --> 00:06:38,969 productive. So thanks for sharing that 131 00:06:38,969 --> 00:06:40,593 personal journey that you have so far. 132 00:06:41,193 --> 00:06:44,778 And I know the books that you've written is something about AI, 133 00:06:44,778 --> 00:06:46,846 right? So probably you can share a 134 00:06:46,846 --> 00:06:49,546 little bit of background why you wrote those two books. 135 00:06:49,546 --> 00:06:52,496 Like what kind of things that you actually want to convey by 136 00:06:52,496 --> 00:06:56,308 writing these two books. So I think my first book came 137 00:06:56,308 --> 00:07:00,472 out of this frustration of everybody saying that AI was the 138 00:07:00,472 --> 00:07:02,722 future. We should be AI first companies. 139 00:07:02,752 --> 00:07:07,386 And this was, look, I wrote the book before the ChatGPT thing 140 00:07:07,386 --> 00:07:10,822 happened. But ChatGPT was released when I 141 00:07:10,822 --> 00:07:13,597 was finishing the book. So I added stuff about it, but 142 00:07:13,597 --> 00:07:16,966 it was kind of about that time. But what I'm trying to tell you 143 00:07:16,966 --> 00:07:21,422 is that I've been noticing this before, even that wave of AI 144 00:07:21,422 --> 00:07:25,533 hype that we witnessed. And I wanted to tell people how 145 00:07:25,533 --> 00:07:28,543 AI actually worked, you know, how machine learning works and 146 00:07:28,543 --> 00:07:30,731 why it has some issues, let's say. 147 00:07:30,771 --> 00:07:34,378 Um, there were lots of, you know, it was a very broad book. 148 00:07:34,378 --> 00:07:36,638 I even spoke about consciousness or the mind, you know, because 149 00:07:36,638 --> 00:07:40,042 you have all these people even telling you that we're just 150 00:07:40,042 --> 00:07:42,742 building artificial neurons. But in reality, we don't really 151 00:07:42,742 --> 00:07:44,728 even know how neurons work very well. 152 00:07:45,178 --> 00:07:46,828 So how can we actually be building them? 153 00:07:46,828 --> 00:07:51,058 So I thought it was, it's good PR, but it wasn't quite true. 154 00:07:52,018 --> 00:07:56,739 And I wanted to also convey, if you want my experience in the 155 00:07:56,739 --> 00:08:00,978 business of AI, because I've seen so many projects fail that 156 00:08:00,978 --> 00:08:03,358 identified a way in which they fail. 157 00:08:04,108 --> 00:08:07,898 Almost the same way every time, repeatedly the same mistakes. 158 00:08:07,918 --> 00:08:09,718 And I thought, I need to tell this to the world, right? 159 00:08:09,718 --> 00:08:12,388 So it was a kind of an antidote to AI hype. 160 00:08:12,388 --> 00:08:13,798 That was my, the goal of my book. 161 00:08:14,782 --> 00:08:17,206 Right. So, yeah, I remember back then, 162 00:08:17,206 --> 00:08:19,672 I dunno, like 5, 10 years, uh, ago, right? 163 00:08:19,672 --> 00:08:23,733 So there are other fields of AI compared to what we know now, 164 00:08:23,733 --> 00:08:26,405 which is like generative AI, LLM, you know, all the craze 165 00:08:26,405 --> 00:08:29,355 about these kind of AI. But in the past, there are so 166 00:08:29,355 --> 00:08:32,067 many AI technologies as well. And in fact, in your book you 167 00:08:32,067 --> 00:08:34,457 mentioned about two AI booms that actually happened, but 168 00:08:34,457 --> 00:08:37,522 actually it ended up going nowhere in the end. 169 00:08:37,522 --> 00:08:40,746 So maybe tell us a little bit of this history so that people know 170 00:08:40,746 --> 00:08:43,121 about the context and they can relate to what they're 171 00:08:43,121 --> 00:08:45,826 experiencing now. Yeah, so there was an AI boom in 172 00:08:45,826 --> 00:08:49,050 the sixties, which is kind of crazy because computers were 173 00:08:49,050 --> 00:08:51,612 new, right? And they weren't very powerful. 174 00:08:52,066 --> 00:08:54,577 But people were saying all these things, uh, everybody will lose 175 00:08:54,577 --> 00:08:57,342 their jobs in 20 years. You know, this is something that 176 00:08:57,342 --> 00:09:00,174 was said in the sixties. And there was a whole boom 177 00:09:00,174 --> 00:09:04,148 around AI. The idea was to build those sort 178 00:09:04,148 --> 00:09:08,444 of artificial worlds, right? That you could have a world 179 00:09:08,444 --> 00:09:11,931 where you simulated some physical things in a computer, 180 00:09:11,931 --> 00:09:15,070 and people thought that would have an impact, but it didn't, 181 00:09:15,070 --> 00:09:18,390 right, in the business world. So after that, there was what's 182 00:09:18,390 --> 00:09:22,176 known as an AI winter, which means a time where there was a 183 00:09:22,176 --> 00:09:25,296 very little money in AI and very, uh, little enthusiasm 184 00:09:25,296 --> 00:09:28,692 about AI. Then in the eighties, there was 185 00:09:28,692 --> 00:09:31,230 another boom, right? And not millions invested. 186 00:09:31,980 --> 00:09:35,256 And this was about expert systems, which is when you try 187 00:09:35,256 --> 00:09:39,214 to encode all the knowledge of a person as rules inside a 188 00:09:39,214 --> 00:09:40,893 computer. It didn't work well. 189 00:09:40,923 --> 00:09:44,113 It was impractical to do. And there was another collapse, 190 00:09:44,113 --> 00:09:47,855 if you want, of the AI field. Another bust of the bubble. 191 00:09:48,605 --> 00:09:53,690 It also became almost shameful to say that you worked in the AI 192 00:09:53,690 --> 00:09:57,185 field at that point, right? People said, stop saying AI. 193 00:09:57,215 --> 00:09:59,865 They started saying, I work in cognitive something, 'cause they 194 00:09:59,865 --> 00:10:02,255 didn't really want to say AI 'cause it was stigmatized. 195 00:10:03,019 --> 00:10:07,507 Then we had after that period of, let's say, kind of calmness, 196 00:10:07,507 --> 00:10:11,398 there was a much more moderate, if you want, boom of AI, uh, 197 00:10:11,398 --> 00:10:13,260 that started to happen in the 2000s. 198 00:10:13,689 --> 00:10:17,809 In 2010, it intensified, right? And that's the one that kind of 199 00:10:17,809 --> 00:10:20,834 I lived through. There was a difference here 200 00:10:20,834 --> 00:10:23,369 though, which is that AI in the form of machine learning. 201 00:10:23,369 --> 00:10:25,229 So this was the machine learning boom, right? 202 00:10:25,229 --> 00:10:27,519 The computer learns how to do stuff from data. 203 00:10:28,029 --> 00:10:32,146 And the difference is that this time AI is successfully used in 204 00:10:32,146 --> 00:10:33,899 commercial applications for the first time. 205 00:10:34,649 --> 00:10:37,028 You have your Netflix recommendations and there's AI 206 00:10:37,028 --> 00:10:40,307 there because the computer's learning from your behavior and 207 00:10:40,307 --> 00:10:44,993 also what other users that are similar to you like, and then 208 00:10:44,993 --> 00:10:46,529 you get personalized recommendations. 209 00:10:46,859 --> 00:10:49,967 AI will sort search results when you're shopping for a holiday, 210 00:10:49,967 --> 00:10:52,029 when you're searching for things on Google. 211 00:10:52,259 --> 00:10:56,009 So AI has been with us for a while, and it's been used in a 212 00:10:56,183 --> 00:10:57,923 commercially successful way, right? 213 00:10:58,849 --> 00:11:02,727 And that's when I thought people were exaggerating about the 214 00:11:02,727 --> 00:11:05,639 power of AI already. ' Cause they were saying we need to use 215 00:11:05,639 --> 00:11:06,709 it for absolutely everything, right? 216 00:11:07,399 --> 00:11:11,371 And then ChatGPT happened. That's kind of the a new wave, 217 00:11:11,371 --> 00:11:14,257 yeah. Yeah, I thought there was gonna 218 00:11:14,257 --> 00:11:16,637 be a sort of AI winter again before ChatGPT. 219 00:11:16,657 --> 00:11:19,912 And there were signs of it. There were some industries that 220 00:11:19,912 --> 00:11:22,522 were struggling, especially the self-driving car industry, for 221 00:11:22,522 --> 00:11:25,267 example, that was doing really, really badly and still is. 222 00:11:26,017 --> 00:11:28,357 But then with ChatGPT, everyone forgot about that, you know? 223 00:11:28,357 --> 00:11:32,812 And it was like the most impressive boom that we could 224 00:11:32,812 --> 00:11:36,738 ever imagine, yeah. Yeah, so I think, I feel that, 225 00:11:36,738 --> 00:11:40,042 you know, all these hype about ML, I experienced it maybe back 226 00:11:40,042 --> 00:11:44,183 then 20- , I dunno, 2010, 2015, those kind of era where you 227 00:11:44,183 --> 00:11:47,113 start having like, I dunno, vision, OCR, those kind of 228 00:11:47,113 --> 00:11:48,516 things. Recommendation system also 229 00:11:48,516 --> 00:11:52,476 starts to appear. And then maybe a few cloud 230 00:11:52,476 --> 00:11:55,379 computing also productize their AI services, right? 231 00:11:55,379 --> 00:11:58,079 So I think that's when some of the craze about AI, you know, 232 00:11:58,079 --> 00:12:00,758 people are trying to use that. Especially also a lot of 233 00:12:00,758 --> 00:12:03,467 startups, you know, cool disruptors, so to speak, right? 234 00:12:03,467 --> 00:12:05,832 They wanna also implement AI use cases. 235 00:12:06,072 --> 00:12:09,162 And I think, yeah, this time is really so much different. 236 00:12:09,162 --> 00:12:11,592 You know, the ChatGPT, the generative AI era. 237 00:12:12,072 --> 00:12:15,252 So what do you think makes it such a craze now? 238 00:12:15,282 --> 00:12:18,252 Because, uh, you know, you have studied the history before. 239 00:12:18,642 --> 00:12:21,606 Um, but this time it seems so real and so different and people 240 00:12:21,606 --> 00:12:23,712 are even talking about AGI and all that stuff. 241 00:12:23,772 --> 00:12:24,933 I think there are a number of things. 242 00:12:24,933 --> 00:12:29,151 So the first one is that this new generation of AI is used by 243 00:12:29,151 --> 00:12:32,553 the general public directly. Just go into ChatGPT and use it. 244 00:12:33,423 --> 00:12:35,253 The previous generation wasn't like that. 245 00:12:35,463 --> 00:12:40,276 You know, as I said, there's AI recommending you movies on 246 00:12:40,276 --> 00:12:41,883 Netflix, but you don't get to see any of that. 247 00:12:41,883 --> 00:12:46,130 You don't use it yourself. But now with ChatGPT, you are 248 00:12:46,130 --> 00:12:50,016 using it directly. So people have access to AI in a 249 00:12:50,016 --> 00:12:53,164 much more straightforward way. But also, it's conversational 250 00:12:53,164 --> 00:12:55,684 AI. And that's a big difference too. 251 00:12:56,044 --> 00:12:59,367 And there's so much fascination around having a conversation 252 00:12:59,367 --> 00:13:03,505 with a machine. And it's also one of the most 253 00:13:03,505 --> 00:13:06,532 difficult tasks historically. The whole, the Turing test is 254 00:13:06,532 --> 00:13:08,737 that. It's like can you build an AI 255 00:13:08,737 --> 00:13:11,820 that is so powerful that you can't tell whether it's a human 256 00:13:11,820 --> 00:13:14,434 or not? And this test is a conversation 257 00:13:14,434 --> 00:13:16,604 where you chat with an AI, right? 258 00:13:16,624 --> 00:13:20,724 So conversation is fascinating and it has so many applications, 259 00:13:20,724 --> 00:13:24,670 and I think that's also a crucial difference with previous 260 00:13:24,670 --> 00:13:27,374 generation of AI. Yeah. 261 00:13:27,524 --> 00:13:30,984 And I find the barrier for you to use it is so much easier now, 262 00:13:30,984 --> 00:13:32,774 right? So you can just go to a website 263 00:13:32,774 --> 00:13:34,724 or even embed it in your phone these days, right? 264 00:13:34,964 --> 00:13:38,187 And even you can speak to it the assistant way, uh, just like 265 00:13:38,187 --> 00:13:40,639 what you mentioned, right? I think, yeah, so many people 266 00:13:40,639 --> 00:13:42,975 just try, and they believe it's pretty smart. 267 00:13:43,515 --> 00:13:45,585 Your book is titled Smart But It's Dumb. 268 00:13:45,585 --> 00:13:48,381 So why do you call it? So I guess that's the first 269 00:13:48,381 --> 00:13:50,945 thing about AI, right? Because people think it's smart, 270 00:13:50,945 --> 00:13:53,261 but actually, maybe there's another side to it that we don't 271 00:13:53,261 --> 00:13:56,060 know. I think that that was pretty 272 00:13:56,060 --> 00:14:00,660 much the thesis of my book, which is that AI works really 273 00:14:00,660 --> 00:14:04,056 nicely and it's impressive, until it's not. 274 00:14:04,086 --> 00:14:07,661 Because you discover... I called it at that time an epic 275 00:14:07,661 --> 00:14:10,191 mistake. Now it's called a hallucination, 276 00:14:10,191 --> 00:14:11,646 right? People, that's the name that 277 00:14:11,646 --> 00:14:14,246 people have adopted for it. But I've been warning people 278 00:14:14,246 --> 00:14:17,682 about this for a very long time, which is that everything's nice 279 00:14:17,682 --> 00:14:21,936 with AI until it makes a mistake that a human would never make. 280 00:14:22,806 --> 00:14:25,968 Ever. And this reveals that it wasn't 281 00:14:25,968 --> 00:14:28,926 actually smart, right? And those things again are, 282 00:14:28,926 --> 00:14:32,158 they're very difficult to quantify and to even, you know, 283 00:14:32,158 --> 00:14:34,625 define. But intuitively, all of a sudden 284 00:14:34,625 --> 00:14:37,247 you're like, okay, this is really, really dumb. 285 00:14:38,340 --> 00:14:41,126 And I've been observing this for a long time and I thought this 286 00:14:41,126 --> 00:14:45,177 was, for example, the thing that would kill the self-driving car 287 00:14:45,177 --> 00:14:48,177 industry. And it is the thing that did 288 00:14:48,177 --> 00:14:51,486 kill the, it's almost dead. Like, uh, Cruise cars, you know. 289 00:14:51,486 --> 00:14:53,938 The brand Cruise that builds self-driving cars, they have 290 00:14:53,938 --> 00:14:57,966 more people in a control room than self-driving cars. 291 00:14:58,086 --> 00:15:01,801 And they operate them remotely. They don't really operate, they 292 00:15:01,801 --> 00:15:04,704 react to problems, right? And they happen so often they 293 00:15:04,704 --> 00:15:06,336 need more people than cars, right? 294 00:15:06,636 --> 00:15:08,946 The other company, Waymo, they also have a control room and 295 00:15:08,946 --> 00:15:12,288 they intervene every three to five miles of driving manually, 296 00:15:12,288 --> 00:15:14,639 right? And I knew this was gonna 297 00:15:14,639 --> 00:15:16,010 happen. I've been saying it for years. 298 00:15:16,040 --> 00:15:19,970 And the problem is that machine learning learns by repetition, 299 00:15:19,970 --> 00:15:24,632 learns by example, right? So you need to show examples of 300 00:15:24,632 --> 00:15:27,830 situations and then the computer kind of interpolates. 301 00:15:28,472 --> 00:15:31,445 If it seemed to similar examples and there's something in 302 00:15:31,445 --> 00:15:35,592 between, it can make deductions, but it's not very good at 303 00:15:35,592 --> 00:15:38,958 processing situations that are far from anything the computer 304 00:15:38,958 --> 00:15:42,727 has had access to the data. And when you're driving, there's 305 00:15:42,727 --> 00:15:46,701 so many things that can happen that are completely out of the 306 00:15:46,701 --> 00:15:48,353 ordinary. And for some reason though, we 307 00:15:48,353 --> 00:15:50,882 don't really understand, humans are very good at that. 308 00:15:51,332 --> 00:15:53,926 We seem to have these internalized models of the world 309 00:15:53,926 --> 00:15:57,196 that we use to process things, to analyze things that, you 310 00:15:57,196 --> 00:15:59,726 know, you see something totally different you've never seen but 311 00:15:59,726 --> 00:16:03,442 you know how to react to it. And with machine learning, it's 312 00:16:03,442 --> 00:16:08,110 notoriously bad at that, right? And I said, this is going to be 313 00:16:08,110 --> 00:16:09,657 bad for the self-driving car industry. 314 00:16:09,921 --> 00:16:14,361 And that's what's been making the adoption of AI so hard now. 315 00:16:14,391 --> 00:16:17,511 Even now, generative AI in the enterprise world, right? 316 00:16:17,873 --> 00:16:21,634 Or even to code, to code in a serious way. 317 00:16:21,634 --> 00:16:24,725 People say, oh, it does 80% of the job, but then I have to go 318 00:16:24,725 --> 00:16:28,901 and debug the thing that AI created, and it's causing me 319 00:16:28,901 --> 00:16:31,271 more trouble than if I'd done it myself, you know. 320 00:16:31,271 --> 00:16:34,610 Not always, but there's a, and it's again, these 321 00:16:34,610 --> 00:16:37,285 hallucinations. And that's why I thought smart 322 00:16:37,285 --> 00:16:40,921 until it's dumb is good way to summarize this, right? 323 00:16:41,221 --> 00:16:44,821 Kind of describes the whole situation in a very short 324 00:16:44,821 --> 00:16:46,015 sentence. Yeah. 325 00:16:47,005 --> 00:16:49,645 Yeah, I think almost everyone, including software engineers, 326 00:16:49,645 --> 00:16:51,641 right? I mean, we are, I mean I 327 00:16:51,641 --> 00:16:54,045 perceive us as someone who understands how a computer works 328 00:16:54,045 --> 00:16:56,785 and maybe we know about logical thinking and things like that. 329 00:16:56,785 --> 00:17:00,395 Even us are still kind of like impressed sometimes when you use 330 00:17:00,395 --> 00:17:02,692 coding assistants. You know, how come it can 331 00:17:02,692 --> 00:17:04,645 generate a code that seems to work? 332 00:17:04,645 --> 00:17:06,586 I mean, it works. There's a vibe coding as well, 333 00:17:06,586 --> 00:17:08,214 right, these days. Uh, it seems to work. 334 00:17:08,214 --> 00:17:11,179 But yeah, there are times where we think it would work, but 335 00:17:11,179 --> 00:17:14,476 sometimes it just fail epically maybe, uh, borrowing the word 336 00:17:14,476 --> 00:17:17,082 that you mentioned. And many, many catastrophic 337 00:17:17,082 --> 00:17:19,564 mistakes actually happening as well, like deleting production 338 00:17:19,564 --> 00:17:22,588 database and things like that. So definitely there's something 339 00:17:22,588 --> 00:17:27,060 that we, like I personally, even though I use AI a lot, I don't 340 00:17:27,060 --> 00:17:29,854 understand the fundamentals. Like I don't understand how all 341 00:17:29,854 --> 00:17:31,696 these LLM, generative AI actually works. 342 00:17:31,966 --> 00:17:35,296 Maybe it's also a good time for us to understand a little bit, 343 00:17:35,296 --> 00:17:38,032 uh, about LLM, right? Yeah, yeah. 344 00:17:38,422 --> 00:17:42,126 Yeah, sure, so we need to... um, a language model, very simple 345 00:17:42,126 --> 00:17:44,858 computer program that tries to predict the next word based on 346 00:17:44,858 --> 00:17:47,204 the previous word. Very, very simple. 347 00:17:47,594 --> 00:17:51,386 And if you wanted to produce an entire sentence, you just make 348 00:17:51,386 --> 00:17:54,421 it eats its own output. So you generate one word or a 349 00:17:54,421 --> 00:17:57,253 token, should be a piece of a word from a vocabulary, it 350 00:17:57,253 --> 00:18:01,031 generates a piece of text, you add it to the prompt, then you 351 00:18:01,031 --> 00:18:02,801 run it through the program again, right? 352 00:18:03,101 --> 00:18:05,651 That's all it does. Anything else is a sort of 353 00:18:05,651 --> 00:18:09,845 illusion built on top. So for example, when AI seems to 354 00:18:09,845 --> 00:18:13,524 search the web, there's actually another program, a wrapper 355 00:18:13,524 --> 00:18:17,310 around the LLM, which intercepts an instruction that says please 356 00:18:17,310 --> 00:18:20,295 go search the web. And then you go and you search 357 00:18:20,295 --> 00:18:21,873 the web with another computer program. 358 00:18:22,045 --> 00:18:25,193 And then you gather some results and you put them in your prompt 359 00:18:25,193 --> 00:18:28,675 and then you continue. The user doesn't see this, 360 00:18:28,675 --> 00:18:30,750 right? But under the hood it's just a 361 00:18:30,750 --> 00:18:35,927 system to predict the next word. And it was trained on two types 362 00:18:35,927 --> 00:18:39,217 of data. The first one is a huge amount 363 00:18:39,217 --> 00:18:40,825 of data collected from the internet. 364 00:18:41,905 --> 00:18:47,797 So if you want, this learns how to guess the best next word, 365 00:18:47,797 --> 00:18:50,200 according to what internet have said. 366 00:18:50,790 --> 00:18:52,973 There are lots of problems have been solved by people. 367 00:18:52,973 --> 00:18:56,434 The solutions are online. And you can see that this can 368 00:18:56,434 --> 00:18:58,575 take you pretty far but not completely far. 369 00:18:58,665 --> 00:19:01,518 The system will not gonna learn how to do math, for example, 370 00:19:01,518 --> 00:19:05,481 'cause people are not saying online two plus four equals six 371 00:19:05,481 --> 00:19:08,532 with all possible numbers for the AI to actually get it. 372 00:19:09,082 --> 00:19:12,562 So what AI is gonna do is going to create a plausible output. 373 00:19:12,562 --> 00:19:16,862 Something that looks like a solution but not an actual 374 00:19:16,862 --> 00:19:19,192 solution 'cause there's no data to support it. 375 00:19:19,942 --> 00:19:22,552 Same thing with the names of citations of legal cases. 376 00:19:22,552 --> 00:19:26,162 There's this lawyer uses ChatGPT to try to find legal precedence 377 00:19:26,162 --> 00:19:29,096 for a case. And it kind of invents names 378 00:19:29,096 --> 00:19:31,046 that sound good, but they're fake. 379 00:19:31,046 --> 00:19:32,859 And I've seen that a lot of times. 380 00:19:33,662 --> 00:19:40,062 So the second stage in training these AI models was to use 381 00:19:40,062 --> 00:19:44,912 manual data created by humans. So what they did was they asked 382 00:19:44,912 --> 00:19:49,143 people to generate lots of responses, and they ranked them 383 00:19:49,143 --> 00:19:50,615 manually. They said, this is a good 384 00:19:50,615 --> 00:19:51,797 response, this is a bad response. 385 00:19:51,797 --> 00:19:54,234 They did a lot of, you know, trying to make it more 386 00:19:54,234 --> 00:19:55,937 appropriate, not say inappropriate things this way 387 00:19:55,937 --> 00:19:58,397 with people, you know. Paid very little money. 388 00:19:58,397 --> 00:20:02,267 Some people say they hired people in Kenya for $2 an hour 389 00:20:02,669 --> 00:20:05,599 to do a lot of this manual work. And then all this data was fed 390 00:20:05,599 --> 00:20:07,939 back into the machine. This is called reinforcement 391 00:20:07,939 --> 00:20:11,121 learning with human feedback to improve the models, make them 392 00:20:11,121 --> 00:20:14,556 more aligned, made them better at solving well-known problems, 393 00:20:14,556 --> 00:20:17,161 right? And that's how they're trying to 394 00:20:17,161 --> 00:20:19,007 fix the hallucinations and all these problems. 395 00:20:19,637 --> 00:20:23,121 But there's always like a long tail of problems that never 396 00:20:23,121 --> 00:20:25,422 disappears. But yeah, that's the essence of 397 00:20:25,422 --> 00:20:27,114 how... And another thing I want to say, 398 00:20:27,114 --> 00:20:29,937 why, why now? Why, why does... because the way 399 00:20:29,937 --> 00:20:33,440 this computer program works is different from what people were 400 00:20:33,440 --> 00:20:36,110 doing before when they created language models which already 401 00:20:36,110 --> 00:20:38,741 existed. In the past, they were using 402 00:20:38,741 --> 00:20:41,945 something called an LSTM which is a very naive type of 403 00:20:41,945 --> 00:20:44,128 technology, I think. I was always a bit skeptical of 404 00:20:44,128 --> 00:20:45,641 it. 'Cause what it does is processes 405 00:20:45,641 --> 00:20:49,513 one word at a time, and it tries to kind of summarize the entire 406 00:20:49,513 --> 00:20:55,483 context up to now in one little vector of numbers that tries to 407 00:20:55,483 --> 00:20:59,489 express what's been said so far, which is a huge compression of 408 00:20:59,489 --> 00:21:02,285 all this information into a tiny little thing. 409 00:21:02,285 --> 00:21:04,565 And then you use that to predict the next word. 410 00:21:04,865 --> 00:21:07,355 But you've pretty much, you've forgotten a lot of stuff. 411 00:21:07,355 --> 00:21:11,209 It's very, very hard. So what they invented was this 412 00:21:11,209 --> 00:21:14,110 so-called transformer architecture that takes the 413 00:21:14,110 --> 00:21:18,860 whole context and tries to represent the meaning of every 414 00:21:18,860 --> 00:21:22,170 token in the context simultaneously. 415 00:21:22,630 --> 00:21:26,943 They try to disambiguate the meaning of tokens by kind of 416 00:21:26,943 --> 00:21:30,657 cross-referencing them, and then it predicts the next word from a 417 00:21:30,657 --> 00:21:33,331 really, really rich representation of the context. 418 00:21:33,331 --> 00:21:35,761 And this is what changed from previous technologies. 419 00:21:36,421 --> 00:21:38,851 And this is something we've seen a lot in AI, which is that they, 420 00:21:38,911 --> 00:21:43,875 somebody invents a new approach and that's when performance is 421 00:21:43,875 --> 00:21:46,126 unleashed. It happened with convolutional 422 00:21:46,126 --> 00:21:48,241 networks for image analysis, right? 423 00:21:48,515 --> 00:21:52,043 And now transformer architecture for text generation. 424 00:21:52,922 --> 00:21:57,175 Yeah, so it kind of is in a nutshell how AI works. 425 00:21:57,474 --> 00:22:01,470 Very simple principles, very powerful, but also not all 426 00:22:01,470 --> 00:22:03,178 powerful. Right. 427 00:22:03,358 --> 00:22:06,378 And when people refer to generative AI, is it referring 428 00:22:06,378 --> 00:22:10,162 to the same thing as LLM or is it something slightly different? 429 00:22:10,448 --> 00:22:13,627 Well, that's complicated, first of all, because in academic 430 00:22:13,627 --> 00:22:18,030 circles, generative AI does not mean what people usually mean by 431 00:22:18,030 --> 00:22:20,680 generative AI. So forgetting about the 432 00:22:20,680 --> 00:22:23,320 technical statistical definition, just generative AI 433 00:22:23,320 --> 00:22:27,500 means AI to generate things. So it could be LLMs when we're 434 00:22:27,500 --> 00:22:30,660 talking about language, but it could be also images, right? 435 00:22:30,900 --> 00:22:33,726 And that's not an LLM necessarily, although you may be 436 00:22:33,726 --> 00:22:36,333 typing in something or, you know, when you're transferring 437 00:22:36,333 --> 00:22:39,142 this style from an image to another, you could call the 438 00:22:39,142 --> 00:22:40,547 generative AI, but there's no language. 439 00:22:40,577 --> 00:22:44,070 When you're typing in, there is a language model somewhere in 440 00:22:44,070 --> 00:22:45,527 between, but you're generating an image. 441 00:22:45,797 --> 00:22:49,455 But what I would say is it's mostly about generating new 442 00:22:49,455 --> 00:22:52,935 content based on some cues or some prompts that we give to the 443 00:22:52,935 --> 00:22:55,223 machine. Thank you for the clarification, 444 00:22:55,223 --> 00:22:57,848 right? So every time I read about the 445 00:22:57,848 --> 00:23:00,574 basics of LLM, right, the primitive understanding is that, 446 00:23:00,574 --> 00:23:02,932 yeah, it tries to predict something based on a given 447 00:23:02,932 --> 00:23:05,182 input, right? And you just multiply it with so 448 00:23:05,182 --> 00:23:07,702 many inputs and maybe you train, you kind of like do 449 00:23:07,702 --> 00:23:08,832 reinforcement learning and all that. 450 00:23:09,252 --> 00:23:11,202 But somehow it just works magically, you know. 451 00:23:11,202 --> 00:23:14,194 If you use some kind of assistant or maybe ChatGPT, they 452 00:23:14,194 --> 00:23:16,946 will say, thinking, right? Thinking and then gives you some 453 00:23:16,946 --> 00:23:20,360 explanation of what it tries to do, which if you read about it 454 00:23:20,360 --> 00:23:23,613 seems very natural, like a human thinking and trying to deduce 455 00:23:23,613 --> 00:23:25,868 stuff. And even we use it for coding. 456 00:23:25,868 --> 00:23:29,108 Like I'm still amazed kind of like how come it can produce 457 00:23:29,108 --> 00:23:32,786 code in a certain framework, in a certain language, that mostly, 458 00:23:32,786 --> 00:23:36,002 mostly, uh, works syntactically correct and with a good output 459 00:23:36,002 --> 00:23:39,320 and things like that. Like maybe explain us a little 460 00:23:39,320 --> 00:23:41,162 bit. Like I'm still kind of like 461 00:23:41,162 --> 00:23:44,574 don't understand in between how come it can be so powerful and 462 00:23:44,574 --> 00:23:48,303 looks really smart. I think that internally, those 463 00:23:48,303 --> 00:23:52,347 computer programs, right, the LLMs, they have states. 464 00:23:52,617 --> 00:23:57,050 They're called hidden states. We can think of a hidden state 465 00:23:57,050 --> 00:24:00,876 as a sort of abstraction that we don't necessarily understand 466 00:24:00,876 --> 00:24:04,736 what it is. We let the computer learn its 467 00:24:04,736 --> 00:24:08,417 own abstractions, right? So for example, the LLM may 468 00:24:08,417 --> 00:24:12,376 learn that to produce or to make good predictions about code. 469 00:24:12,406 --> 00:24:13,726 'Cause that's what it's trying to do. 470 00:24:14,746 --> 00:24:19,336 It needs to have some sort of internal state that tries to 471 00:24:19,336 --> 00:24:21,436 produce syntactically correct code. 472 00:24:21,804 --> 00:24:25,673 As in it will maybe count the number of opening parentheses, 473 00:24:25,673 --> 00:24:28,621 you know, so that it matches the kind of the closing parentheses. 474 00:24:29,071 --> 00:24:31,231 So that's what made it, makes it really powerful. 475 00:24:31,231 --> 00:24:35,733 There are many layers stacked on top of each other that have 476 00:24:35,733 --> 00:24:42,747 hidden states that try to, when the model is trained, these 477 00:24:42,747 --> 00:24:46,454 hidden states configure themselves if you want, in the 478 00:24:46,454 --> 00:24:48,456 best possible way to solve the problem. 479 00:24:49,002 --> 00:24:50,922 And because there are so many layers of them, we can't really 480 00:24:50,922 --> 00:24:53,850 understand where they are, if we look because it's gigantic, 481 00:24:53,850 --> 00:24:56,260 right? But I would say, because some 482 00:24:56,260 --> 00:24:59,686 people, they become a little too critical of AI and they said 483 00:24:59,686 --> 00:25:03,411 it's stochastic parrot. It just imitates or copy paste 484 00:25:03,411 --> 00:25:06,705 stuff from the data, the training data. 485 00:25:06,735 --> 00:25:10,013 Not really, right? What it does is it uses the 486 00:25:10,013 --> 00:25:13,455 training data to try to configure those internal hidden 487 00:25:13,455 --> 00:25:16,449 states. Initially, uh, I remember asking 488 00:25:16,449 --> 00:25:20,394 ChatGPT, does an Anaconda fit in a shopping mall, right? 489 00:25:20,454 --> 00:25:23,438 And it told me that it didn't because it's such a large 490 00:25:23,438 --> 00:25:26,640 animal, right? What this shows you that there 491 00:25:26,640 --> 00:25:30,153 was no hidden state that properly represented the size of 492 00:25:30,153 --> 00:25:33,264 things or what things are big or what big means, right? 493 00:25:33,680 --> 00:25:37,416 It was still producing good text in the sense that it sounded 494 00:25:37,416 --> 00:25:41,301 correct, because its hidden states had properly represented 495 00:25:41,301 --> 00:25:45,499 the grammar of English, right? But not necessarily the sizes of 496 00:25:45,499 --> 00:25:47,991 things. But then this became quite viral 497 00:25:47,991 --> 00:25:50,074 and as they improved the algorithms, they probably 498 00:25:50,074 --> 00:25:53,598 generated manual data. I have to believe that they did 499 00:25:53,598 --> 00:25:55,826 that probably somewhere where people asking questions about 500 00:25:55,826 --> 00:25:59,027 the sizes of things and manually ranking the answers and that was 501 00:25:59,027 --> 00:26:03,057 fed back to the LLM. And at some point, the LLM must 502 00:26:03,057 --> 00:26:06,854 have created an internal hidden state that tries to represent 503 00:26:06,854 --> 00:26:09,912 the things that are big in general, right? 504 00:26:09,912 --> 00:26:13,452 And when things fit into other things, stuff like that. 505 00:26:13,842 --> 00:26:16,502 We can't really know what it is 'cause it could use shortcuts. 506 00:26:17,142 --> 00:26:20,607 There was a famous case of this image recognition algorithm that 507 00:26:20,607 --> 00:26:24,809 or model that failed to detect a cow on the beach. 508 00:26:25,339 --> 00:26:28,109 And the reason was that a cow was not on grass. 509 00:26:28,342 --> 00:26:31,303 So the algorithm had actually learned that a cow is this thing 510 00:26:31,303 --> 00:26:34,305 that looks like a cow and the grass underneath, because all of 511 00:26:34,305 --> 00:26:37,345 the images and the training data had grass, right? 512 00:26:37,675 --> 00:26:39,655 So it didn't really learn what a cow was. 513 00:26:39,775 --> 00:26:42,635 So they can, and that this also causes hallucinations, right? 514 00:26:42,635 --> 00:26:45,535 The AI may not really learn what we want from it. 515 00:26:46,055 --> 00:26:49,069 However it does have hidden states that try to represent 516 00:26:49,069 --> 00:26:52,580 things in an, with a higher level of abstraction, I would 517 00:26:52,580 --> 00:26:54,489 say. And because they're stacked on 518 00:26:54,489 --> 00:26:57,317 top of each other, you can combine them, right? 519 00:26:57,317 --> 00:27:01,565 So there's a first layer where it tries to represent some low 520 00:27:01,565 --> 00:27:03,797 level things, right? Is this a noun? 521 00:27:03,797 --> 00:27:07,027 Is this a verb, right? At the next level, it could 522 00:27:07,027 --> 00:27:10,905 represent, is this a noun that represents a big object or a 523 00:27:10,905 --> 00:27:14,556 small object, right? And after lots and lots of 524 00:27:14,556 --> 00:27:17,268 layers, you get a very, very good representation of the 525 00:27:17,268 --> 00:27:20,452 meaning of things. And I think, yeah, the key to 526 00:27:20,452 --> 00:27:23,949 the power of this is all this hidden stuff that goes inside 527 00:27:23,949 --> 00:27:27,185 the machine. So I understand like sometimes 528 00:27:27,185 --> 00:27:29,387 machine learning, you can't really explain what is happening 529 00:27:29,387 --> 00:27:32,104 because it goes through, I don't know, like thousands or maybe 530 00:27:32,104 --> 00:27:34,456 millions iterations, and that's why probably these hidden states 531 00:27:34,456 --> 00:27:37,531 are kind of like embedded inside the model, so to speak, right? 532 00:27:37,711 --> 00:27:41,101 And maybe now it's also a good time to understand about what is 533 00:27:41,101 --> 00:27:43,741 called a model. Like people are calling 534 00:27:43,741 --> 00:27:47,581 foundational models, you know, OpenAI, Gemini, Claude, 535 00:27:47,581 --> 00:27:49,498 whatever. I dunno how many foundational 536 00:27:49,498 --> 00:27:52,185 models are there. So maybe if you can explain what 537 00:27:52,185 --> 00:27:55,417 is foundational model? Okay, let's start with model. 538 00:27:56,187 --> 00:27:59,584 A model is essentially a computer program that tries to 539 00:27:59,584 --> 00:28:01,822 predict something. So it's the essence of this AI. 540 00:28:02,736 --> 00:28:05,282 There are different reasons for the use of the word model, we 541 00:28:05,282 --> 00:28:07,947 could say it's trying to model the world, but I'm a bit more 542 00:28:07,947 --> 00:28:11,269 practical with that. I don't like to use the word 543 00:28:11,269 --> 00:28:13,221 program, 'cause there are two programs in AI. 544 00:28:13,761 --> 00:28:16,971 One is the program that trains the other program. 545 00:28:17,241 --> 00:28:22,279 You have a program, right, which is this model which an AI 546 00:28:22,279 --> 00:28:25,697 engineer will configure, but they leave holes in it. 547 00:28:25,727 --> 00:28:28,753 Things that are undefined that are called parameters, which are 548 00:28:28,753 --> 00:28:32,494 numbers, right? So you create a sort of template 549 00:28:32,494 --> 00:28:36,305 of the computer program, and then you need to run your 550 00:28:36,305 --> 00:28:39,875 training, which is how to scan all this data to try to fill in 551 00:28:39,875 --> 00:28:42,707 those blanks in the template. And that's another program. 552 00:28:42,707 --> 00:28:45,047 And if we use the word program, we can, we get confused. 553 00:28:45,047 --> 00:28:48,434 So which one is which, right? So typically you'll use the word 554 00:28:48,434 --> 00:28:51,367 model to represent the final results of your training, which 555 00:28:51,367 --> 00:28:53,927 is this program with all the blanks filled in, right? 556 00:28:53,987 --> 00:28:57,111 And then you use the word training algorithm or something 557 00:28:57,111 --> 00:29:00,911 like that to explain the alg-, the program that actually does 558 00:29:00,911 --> 00:29:03,500 the training, right? So have a computer program 559 00:29:03,500 --> 00:29:04,712 building another computer program. 560 00:29:04,742 --> 00:29:09,014 The last one is the model. And I think with foundational 561 00:29:09,014 --> 00:29:13,805 models, people mean very general purpose LLMs that can do lots of 562 00:29:13,805 --> 00:29:16,705 things, right? And they can be used as building 563 00:29:16,705 --> 00:29:20,558 blocks for other AI. So let's say you want to build 564 00:29:20,558 --> 00:29:23,281 some, an app that is based on AI. 565 00:29:23,996 --> 00:29:28,486 You may connect using an API to a foundational model, right? 566 00:29:29,206 --> 00:29:32,689 Or you may take the foundational model that is open source and 567 00:29:32,689 --> 00:29:36,961 fine tune it to your own data so it works better on your specific 568 00:29:36,961 --> 00:29:39,284 problem, right? The foundational model would be 569 00:29:39,284 --> 00:29:41,550 very, very generic. And then you have your specific 570 00:29:41,550 --> 00:29:44,318 model to do what you want. Right. 571 00:29:44,658 --> 00:29:47,938 So I think I'm always very fascinated by people who are 572 00:29:48,008 --> 00:29:49,393 producing this foundational model. 573 00:29:49,393 --> 00:29:52,197 I can't really imagine how they train it because it's such a 574 00:29:52,197 --> 00:29:54,523 general purpose. You can use it for language, you 575 00:29:54,523 --> 00:29:57,511 can use it for translation, math, whatever people wanna try, 576 00:29:57,511 --> 00:29:59,914 right? I think it seems to be quite 577 00:29:59,914 --> 00:30:02,881 smart and powerful. So these days, there are new 578 00:30:02,881 --> 00:30:05,511 techniques that people are trying out, you know, things 579 00:30:05,511 --> 00:30:09,933 like RAG and also agentic AI. So maybe if you can elaborate 580 00:30:09,933 --> 00:30:12,878 these also shortly and maybe what other things that are new 581 00:30:12,878 --> 00:30:14,523 that probably we are not familiar with. 582 00:30:15,204 --> 00:30:18,016 Yeah. RAG means retrieval augmented 583 00:30:18,016 --> 00:30:20,004 generation. It sounds very fancy, but it's 584 00:30:20,004 --> 00:30:21,366 not. It's essentially a following. 585 00:30:21,366 --> 00:30:23,277 You have, let's say in your company, you have a database 586 00:30:23,277 --> 00:30:25,746 with a lot of your own private data, right? 587 00:30:26,224 --> 00:30:29,650 And you want to search, using, use AI to search or analyze that 588 00:30:29,650 --> 00:30:32,251 data. So what you do when the user 589 00:30:32,251 --> 00:30:35,488 queries something, they search for something, or they ask in a 590 00:30:35,488 --> 00:30:38,827 prompt for something. You run a separate algorithm to 591 00:30:38,827 --> 00:30:41,124 go and find relevant documents that could help. 592 00:30:42,049 --> 00:30:45,529 And then you insert the text of those documents in the prompt. 593 00:30:45,709 --> 00:30:48,202 The user doesn't see this, but you say, okay, these are the 594 00:30:48,202 --> 00:30:50,244 documents. So you need a large context 595 00:30:50,244 --> 00:30:52,627 window. You need, uh, a model that 596 00:30:52,627 --> 00:30:55,369 allows a lot of or very long prompts. 597 00:30:55,729 --> 00:30:58,549 You insert all this data and then you add instructions. 598 00:30:58,549 --> 00:31:02,233 You say, you know, answer the original question referring to 599 00:31:02,233 --> 00:31:05,834 this data. What this lets you do is not 600 00:31:05,834 --> 00:31:09,337 change the foundational model, not fine tune it to your data, 601 00:31:09,337 --> 00:31:12,694 right? So it's a way to specialize an 602 00:31:12,694 --> 00:31:17,037 AI to your own company data or any specific data without 603 00:31:17,037 --> 00:31:20,101 changing the model. So yeah, that's RAG. 604 00:31:20,101 --> 00:31:23,263 And the challenge is, okay, how do we find that data? 605 00:31:23,633 --> 00:31:25,253 How do we find the relevant stuff? 606 00:31:25,347 --> 00:31:29,051 But a good application of that is what we see with Google AI 607 00:31:29,051 --> 00:31:30,775 results, which tend to be terrible by the way. 608 00:31:30,775 --> 00:31:36,671 But anyway, we get to see AI using websites and analyzing 609 00:31:36,671 --> 00:31:38,971 websites. And the way it probably works 610 00:31:38,971 --> 00:31:42,073 under the hood is that first Google finds relevant websites 611 00:31:42,073 --> 00:31:45,640 using its old algorithms. And then it takes the text in 612 00:31:45,640 --> 00:31:48,802 those websites, puts it inside some prompt somewhere, it says 613 00:31:48,802 --> 00:31:52,597 answer this query using also this information, right? 614 00:31:52,597 --> 00:31:55,087 And then the LLM will generate answers which are then 615 00:31:55,117 --> 00:31:56,767 intercepted and shown to the user. 616 00:31:57,887 --> 00:32:01,464 So that's RAG. And then agentic AI or AI 617 00:32:01,464 --> 00:32:06,308 agents, I feel like there's a new word every day, um, coming 618 00:32:06,308 --> 00:32:08,724 up. And now agentic seems to be the 619 00:32:08,724 --> 00:32:12,633 new word. An AI agent means that there's 620 00:32:12,633 --> 00:32:18,682 an AI model that you use in a pretty autonomous way and you 621 00:32:18,682 --> 00:32:22,991 execute some actions that come out of the model probably in an 622 00:32:22,991 --> 00:32:25,539 automated way. So for example, you create a 623 00:32:25,539 --> 00:32:29,979 little program that uses an LLM to generate some instructions. 624 00:32:30,879 --> 00:32:32,439 And then you use that instruction. 625 00:32:32,739 --> 00:32:36,333 I can imagine, I don't know, imagine an agent that scans your 626 00:32:36,333 --> 00:32:40,589 emails of a corporate person who travels a lot for business and 627 00:32:40,589 --> 00:32:45,365 you tell the LLM whenever you detect that this person wants to 628 00:32:45,365 --> 00:32:48,386 travel somewhere, output this. ' Cause the agent can't really do 629 00:32:48,386 --> 00:32:49,826 it. It's like output an instruction 630 00:32:49,826 --> 00:32:52,731 that says search for flight for these dates in this specific 631 00:32:52,731 --> 00:32:54,986 format. Then you connect that to an API 632 00:32:54,986 --> 00:32:57,881 that goes and searches for the flight, for example, right? 633 00:32:58,264 --> 00:33:01,408 And then you get the results and give them to the LLM and you 634 00:33:01,408 --> 00:33:04,581 say, you know, send an email to this person with a proposal of 635 00:33:04,581 --> 00:33:06,094 this could be your flight, right? 636 00:33:06,424 --> 00:33:10,296 So essentially automate workflows and maybe even 637 00:33:10,296 --> 00:33:14,800 connecting multiple AI agents. But essentially every agent is 638 00:33:14,800 --> 00:33:19,838 nothing but an LLM to which you send a fancy prompt. 639 00:33:20,934 --> 00:33:23,464 Right. So I think, um, when reading 640 00:33:23,464 --> 00:33:26,662 your book as well, right? Explaining all these, uh, 641 00:33:26,662 --> 00:33:30,082 fundamentals thing about the AI, LLM, things that we understand 642 00:33:30,082 --> 00:33:33,967 now as like a cool thing, right? So definitely you can understand 643 00:33:33,967 --> 00:33:37,282 some holes that could probably, uh, poke, you poke, right? 644 00:33:37,987 --> 00:33:41,329 And you can start seeing, okay, why sometimes it hallucinates or 645 00:33:41,329 --> 00:33:44,430 it makes mistakes, right? Simply because of the way it is, 646 00:33:44,430 --> 00:33:47,114 I dunno, the fundamentals of how it works, right? 647 00:33:47,114 --> 00:33:50,594 And maybe we can find clever ways of doing things like the 648 00:33:50,594 --> 00:33:52,663 RAG and maybe the wrapper that you mentioned, right? 649 00:33:52,663 --> 00:33:54,488 Don't forget. I think the wrapper is also 650 00:33:54,488 --> 00:33:57,751 very, very powerful, I see. Because, um, you know, you can 651 00:33:57,751 --> 00:34:01,672 tweak the workflow, so to speak, within the way AI replies the 652 00:34:01,672 --> 00:34:05,063 response to you, right? I think the wrapper is kind of 653 00:34:05,063 --> 00:34:08,206 like the intelligence as well that people build on top of the 654 00:34:08,206 --> 00:34:10,879 foundational models. So I think thanks for 655 00:34:10,879 --> 00:34:12,170 highlighting all these fundamentals. 656 00:34:12,170 --> 00:34:14,789 So when people talk about AI these days, people think about, 657 00:34:14,789 --> 00:34:17,605 I need to use AI. Businesses also scrambling so 658 00:34:17,605 --> 00:34:21,143 hard to implement, adopt AI. First maybe adopt like ChatGPT 659 00:34:21,143 --> 00:34:22,877 and all that within their companies. 660 00:34:23,117 --> 00:34:25,708 And also building something smart, you know, using AI. 661 00:34:25,978 --> 00:34:28,195 And I know in your book, you mentioned, you know, there are 662 00:34:28,195 --> 00:34:29,969 some failure modes that could happen. 663 00:34:29,969 --> 00:34:33,495 So maybe tell us how can business or organizations, you 664 00:34:33,495 --> 00:34:36,719 know, start thinking about adopting AI or using AI within 665 00:34:36,719 --> 00:34:38,170 their companies? Yeah. 666 00:34:38,290 --> 00:34:41,024 Well, there are a few things. The first one is that I don't 667 00:34:41,024 --> 00:34:44,237 think the best way to do anything is to have the hammer 668 00:34:44,237 --> 00:34:46,960 and search for nails. You know, saying, what can we do 669 00:34:46,960 --> 00:34:49,600 with AI? I've met companies that created 670 00:34:49,600 --> 00:34:53,406 a whole team to promote the use of AI in the organization, and 671 00:34:53,406 --> 00:34:55,460 it didn't go well because they were very biased. 672 00:34:55,460 --> 00:34:58,402 They were, for the survival of the team, they had to say, this 673 00:34:58,402 --> 00:35:01,486 can be done with AI, right? And they didn't acknowledge the 674 00:35:01,486 --> 00:35:04,729 limitations of AI. In my experience, most AI or 675 00:35:04,729 --> 00:35:08,242 most failed AI projects start with that saying, oh, what can 676 00:35:08,242 --> 00:35:11,212 we do with AI? So I wouldn't do it that way. 677 00:35:11,242 --> 00:35:15,081 Uh, it's good however, to be aware of what AI can do, what 678 00:35:15,081 --> 00:35:18,102 the limitations are, because it's just another tool in the 679 00:35:18,102 --> 00:35:20,110 toolbox, right? So when you find a problem, 680 00:35:20,110 --> 00:35:23,269 something that could be done in a more efficient way in an 681 00:35:23,269 --> 00:35:26,176 organization, you can see if AI will fix it. 682 00:35:26,971 --> 00:35:33,008 I think the best way to use AI is to acknowledge these 683 00:35:33,008 --> 00:35:35,701 hallucinations from the get go or just the limitations in 684 00:35:35,701 --> 00:35:38,856 general of AI. Because if you embed that into 685 00:35:38,856 --> 00:35:41,911 the, your product, you can build a successful product. 686 00:35:43,051 --> 00:35:47,063 If you forget about that, you build a product that people 687 00:35:47,063 --> 00:35:48,629 won't like. Because at one point, it would 688 00:35:48,629 --> 00:35:51,221 be like, you told me this could do this job, but then it 689 00:35:51,221 --> 00:35:53,581 hallucinated, you know, when I least expected it. 690 00:35:53,881 --> 00:35:58,518 So one way to recognize the mistakes, for example, is to 691 00:35:58,518 --> 00:36:02,163 have AI retrieve or point to specific paragraphs of, let's 692 00:36:02,163 --> 00:36:04,623 say, documents and then show you the para-. 693 00:36:04,773 --> 00:36:08,321 So it's so much different when AI interprets stuff than when 694 00:36:08,321 --> 00:36:11,534 you use it to actually point you to something, for example, 695 00:36:11,534 --> 00:36:13,855 right? Because it could help people 696 00:36:13,855 --> 00:36:16,003 search for things more efficiently, right? 697 00:36:16,193 --> 00:36:19,379 And if AI gets it wrong, then you still very quickly you can 698 00:36:19,379 --> 00:36:23,064 see, you can say, okay, AI said this paragraph proves something, 699 00:36:23,064 --> 00:36:25,719 but it actually doesn't, so I'll keep searching. 700 00:36:26,019 --> 00:36:29,637 So I think we need to embed the idea that AI is not perfect in 701 00:36:29,637 --> 00:36:33,271 the way we use it, and that's how you succeed with AI. 702 00:36:33,837 --> 00:36:36,349 And another thing I think is, um, as a developer or in 703 00:36:36,349 --> 00:36:40,583 general, in general we want to use AI when you can describe 704 00:36:40,583 --> 00:36:43,898 your problem very succinctly. Because if you're doing 705 00:36:43,898 --> 00:36:46,846 something so difficult, so custom that it takes you longer 706 00:36:46,846 --> 00:36:50,086 to describe it in the prompt than doing the job, you don't 707 00:36:50,086 --> 00:36:52,123 gain anything. So it needs to be something 708 00:36:52,123 --> 00:36:54,994 that's very generic, very easy to describe, and then it needs 709 00:36:54,994 --> 00:36:56,986 to be easy to validate, you know, the output. 710 00:36:57,490 --> 00:37:00,632 A typical example is you, you know, you're coding and, uh, you 711 00:37:00,632 --> 00:37:03,428 already know how to do this or you've done it before but you 712 00:37:03,428 --> 00:37:07,122 forgot how to do it, right? So you go and ask ChatGPT, how 713 00:37:07,122 --> 00:37:11,306 can I, you know, read a file from S3 using this Boto3 library 714 00:37:11,306 --> 00:37:13,904 and then a very easy to describe. 715 00:37:13,904 --> 00:37:17,049 And then ChatGPT outputs few lines of code which you've used 716 00:37:17,049 --> 00:37:20,501 before but you'd forgotten how. And you can also very quickly 717 00:37:20,501 --> 00:37:22,440 validate if it's working, so it's fine. 718 00:37:23,290 --> 00:37:25,810 Some problems are very difficult to describe, right? 719 00:37:25,810 --> 00:37:28,720 And, uh, and in that case, I think AI is not necessarily 720 00:37:28,720 --> 00:37:32,542 going to save you time. And also sometimes the output is 721 00:37:32,542 --> 00:37:36,634 very difficult to validate. Something you don't know at all 722 00:37:36,634 --> 00:37:39,890 or a very complicated algorithm. And then you're like, you need 723 00:37:39,890 --> 00:37:44,240 to just read someone else's code, someone else being AI. 724 00:37:44,360 --> 00:37:47,514 And, uh, you know, you've read other people's code and 725 00:37:47,514 --> 00:37:49,489 sometimes it's difficult to follow. 726 00:37:49,874 --> 00:37:53,849 So you need to make sure, you know, to use AI in those easy to 727 00:37:53,849 --> 00:37:55,839 describe, easy to validate scenarios, I think. 728 00:37:56,946 --> 00:37:58,806 Yeah, I think that's, uh, pretty good tips, right? 729 00:37:58,806 --> 00:38:02,637 Because I still think that companies try so hard to 730 00:38:02,637 --> 00:38:05,090 implement AI for smart products, right? 731 00:38:05,420 --> 00:38:08,011 So especially these days, like, I mean, almost every product 732 00:38:08,011 --> 00:38:10,490 embeds something called AI features inside it. 733 00:38:10,513 --> 00:38:13,012 It could be, you know, summarizing something and things 734 00:38:13,012 --> 00:38:16,201 like that, right? But the most classical use case 735 00:38:16,201 --> 00:38:18,861 is actually chatbot, right? The conversational thing. 736 00:38:18,861 --> 00:38:21,351 Because it has been around even before LLM, right? 737 00:38:21,351 --> 00:38:25,014 So I think a lot of chatbots. And now, uh, with LLM, I'm sure 738 00:38:25,014 --> 00:38:27,711 people are still trying hard to create chatbots, customer 739 00:38:27,711 --> 00:38:30,764 support thing. And we have seen many, many 740 00:38:30,764 --> 00:38:34,420 cases where, you know, the agent responds something wrongly or 741 00:38:34,420 --> 00:38:38,694 incorrectly, sometimes even give, you know, wrong result to 742 00:38:38,694 --> 00:38:41,302 the customer such that it mislead them or the company, 743 00:38:41,302 --> 00:38:43,387 right. So I think, definitely there's 744 00:38:43,387 --> 00:38:47,146 this risk of hallucination. How can companies deal with this 745 00:38:47,146 --> 00:38:49,623 hallucination? So is there something within 746 00:38:49,623 --> 00:38:53,694 their solution or within their implementation that they can do 747 00:38:53,694 --> 00:38:56,077 in order to reduce this hallucination from happening? 748 00:38:57,293 --> 00:39:00,503 I mean, you can reduce hallucinations. 749 00:39:00,773 --> 00:39:03,083 There are techniques. You can improve your prompts. 750 00:39:03,143 --> 00:39:07,355 You can fine tune your model if it's not working well with your 751 00:39:07,355 --> 00:39:09,473 own data. There's the whole thing called 752 00:39:09,473 --> 00:39:12,901 chain of thought, which is I tell the AI first, build a 753 00:39:12,901 --> 00:39:16,245 series of steps, a recipe to solve the problem and then solve 754 00:39:16,245 --> 00:39:19,403 it, which some people say works better. 755 00:39:19,553 --> 00:39:21,533 Uh, that's what they mean by reasoning models. 756 00:39:21,533 --> 00:39:25,148 It's a whole charade. It's nothing really new there. 757 00:39:25,148 --> 00:39:27,458 It's more like using this chain of thought thing. 758 00:39:28,104 --> 00:39:30,614 But I would say you can't really eliminate hallucinations and 759 00:39:30,614 --> 00:39:32,154 they happen when you least expect them. 760 00:39:32,154 --> 00:39:37,516 So I would not use AI in the traditional way when 761 00:39:37,516 --> 00:39:39,990 hallucinations matter. Hallucinations don't matter in 762 00:39:39,990 --> 00:39:43,440 some context, right? If you're translating reviews of 763 00:39:43,440 --> 00:39:47,602 hotels on TripAdvisor, nobody cares if they're a little bit 764 00:39:47,602 --> 00:39:49,442 hallucinated than they are. Sometimes it's just the 765 00:39:49,442 --> 00:39:52,459 translation's not very good. But I just wanna know if the 766 00:39:52,459 --> 00:39:54,153 hotel is clean, if it's well located. 767 00:39:54,153 --> 00:39:56,523 I don't need a perfect translation, right? 768 00:39:57,273 --> 00:40:00,849 But you can't have it hallucination in literary 769 00:40:00,849 --> 00:40:03,837 translation, right? And probably in customer service 770 00:40:03,837 --> 00:40:06,273 chatbots, you can't have hallucinations either. 771 00:40:06,273 --> 00:40:10,150 It happened to Air Canada that they, you know, it gave the 772 00:40:10,150 --> 00:40:14,712 wrong policy to a customer and then Air Canada had to honor the 773 00:40:14,712 --> 00:40:17,253 policy because a judge told it to. 774 00:40:17,253 --> 00:40:19,865 It's like, you can't just tell the client in the chatbot, yes, 775 00:40:19,865 --> 00:40:23,033 you can do this, we can get a refund for your flight, but then 776 00:40:23,033 --> 00:40:24,673 no, actually you can't, right? So that's not a thing. 777 00:40:25,293 --> 00:40:31,603 So if I were to work on this, I think I would use the chatbot as 778 00:40:31,603 --> 00:40:35,103 a way to let people find the information they need 779 00:40:35,103 --> 00:40:37,157 officially. So the chatbot can be a nicer 780 00:40:37,157 --> 00:40:40,166 way, let's... you could try to interpret what people say, and 781 00:40:40,166 --> 00:40:44,678 then point them to the right place which is the official 782 00:40:44,678 --> 00:40:47,464 data. And you should be very careful 783 00:40:47,464 --> 00:40:50,483 about not letting the chatbot, you know, interpret results 784 00:40:50,483 --> 00:40:53,113 directly, you know. Dunno exactly how you can do 785 00:40:53,113 --> 00:40:56,989 that, but I would frame it more in a way of how can I help you 786 00:40:56,989 --> 00:40:58,963 find information, right? That kind of thing. 787 00:40:59,326 --> 00:41:04,714 Another thing like I feel like a lot of people are praising AI so 788 00:41:04,714 --> 00:41:08,261 much and the things that they can do, but it makes me wonder 789 00:41:08,261 --> 00:41:10,931 if they just weren't doing things very well in the first 790 00:41:10,931 --> 00:41:13,711 place. And it happens a lot with 791 00:41:13,711 --> 00:41:16,235 customer service. I used to work for a company, it 792 00:41:16,235 --> 00:41:19,813 was a travel company. And I understood the issue with 793 00:41:19,813 --> 00:41:23,311 customer service, because... I never contact, you know, 794 00:41:23,311 --> 00:41:26,643 customer service unless something really odd happened. 795 00:41:27,243 --> 00:41:30,663 Otherwise I just go and try to find the information that I need 796 00:41:30,663 --> 00:41:32,103 without having to contact anyone. 797 00:41:32,313 --> 00:41:33,753 But that's not how everyone does it. 798 00:41:33,753 --> 00:41:37,544 In this travel agency, people, they literally call to say, how 799 00:41:37,544 --> 00:41:40,152 much luggage can I bring with me on the plane? 800 00:41:40,893 --> 00:41:44,253 And this bombarded the customer service people. 801 00:41:44,553 --> 00:41:47,229 And the first thing you imagine is, I'm gonna create a chatbot 802 00:41:47,229 --> 00:41:49,848 for this. But I was wondering maybe the 803 00:41:49,848 --> 00:41:51,083 problem they had was communication. 804 00:41:51,083 --> 00:41:54,513 Maybe a few days before the trip, they should send an email 805 00:41:54,513 --> 00:41:58,473 to people telling them, hey, you know, your flight is in two 806 00:41:58,473 --> 00:42:00,437 days. This is the how much luggage you 807 00:42:00,437 --> 00:42:02,772 can bring. And maybe that will solve the 808 00:42:02,772 --> 00:42:06,149 issue, right? And I think there's a bit of, 809 00:42:06,149 --> 00:42:10,102 you know, thinking AI can fix problems that you maybe could 810 00:42:10,102 --> 00:42:12,674 have fixed. Another thing that doesn't cease 811 00:42:12,674 --> 00:42:17,336 to amaze me is this whole thing of AI is so great for 812 00:42:17,336 --> 00:42:21,211 programming because it eliminates all this boilerplate 813 00:42:21,211 --> 00:42:25,356 code. And my question is, why do you 814 00:42:25,356 --> 00:42:27,884 have any boilerplate code in the first place? 815 00:42:28,184 --> 00:42:33,700 I don't have any in my work. I use libraries, frameworks to 816 00:42:33,700 --> 00:42:36,214 avoid... Every time there's anything that 817 00:42:36,484 --> 00:42:38,194 seems like will be a bit repetitive. 818 00:42:38,684 --> 00:42:40,952 I already don't do it. I'm like, what, what library can 819 00:42:40,952 --> 00:42:44,178 I use where someone has done this and or how can I avoid 820 00:42:44,178 --> 00:42:45,976 this? I mean, some people I've said 821 00:42:45,976 --> 00:42:48,486 this before, and some people told me, oh, but look at Django. 822 00:42:48,486 --> 00:42:50,696 Like when you use Django, you need to, you create a new view 823 00:42:50,696 --> 00:42:53,463 and you need to put the URL here, this there, and that 824 00:42:53,463 --> 00:42:55,622 there. And I'm like, it's literally 825 00:42:55,622 --> 00:42:58,320 three things. Is this what AI is really 826 00:42:58,320 --> 00:43:01,275 helping you with? Like I still don't see why 827 00:43:01,275 --> 00:43:03,921 people have so much boilerplate repetition, right? 828 00:43:04,161 --> 00:43:07,971 So sometimes I wonder is there anything deeper here that maybe 829 00:43:07,971 --> 00:43:11,051 you're not doing well and now AI lets you do it better, but maybe 830 00:43:11,051 --> 00:43:13,281 there was a different way before, right? 831 00:43:13,371 --> 00:43:17,931 Um, I, at one point, I asked ChatGPT for an algorithm to 832 00:43:17,931 --> 00:43:21,307 calculate the position of the sun based on the time of the day 833 00:43:21,307 --> 00:43:24,854 and your location. And it, on repeated executions, 834 00:43:24,854 --> 00:43:27,903 it proposed different things. But one of them, it just 835 00:43:27,903 --> 00:43:30,136 provided a very long algorithm filled with numbers and 836 00:43:30,136 --> 00:43:33,032 trigonometry and it had constants in it like numbers. 837 00:43:34,262 --> 00:43:36,558 And I was like, yeah, I could maybe copy paste this in my own 838 00:43:36,558 --> 00:43:38,670 code. I will never know if this works, 839 00:43:38,670 --> 00:43:42,100 if it makes sense. But there was a much easier 840 00:43:42,100 --> 00:43:44,541 solution, which was like, pip install, sun position 841 00:43:44,541 --> 00:43:47,282 calculator, which is the name of a library that does this, right. 842 00:43:47,832 --> 00:43:49,802 Again, I had already, I'd already done this, right? 843 00:43:49,802 --> 00:43:52,472 So I wasn't trying to get ChatGPT to do it for me. 844 00:43:52,472 --> 00:43:55,072 I was more like, I was curious to see what would happen. 845 00:43:55,417 --> 00:43:58,297 So I then I kind of executed the query again, and ChatGPT did 846 00:43:58,297 --> 00:44:01,658 propose a library at another attempt instead of giving me the 847 00:44:01,658 --> 00:44:04,057 code. But it kind of hallucinated the 848 00:44:04,057 --> 00:44:06,925 way in which the library was used, how you called the 849 00:44:06,925 --> 00:44:09,232 function. So when I tried to run it, it 850 00:44:09,232 --> 00:44:12,731 was, it didn't run, right? I was like all this effort, when 851 00:44:12,731 --> 00:44:15,715 in reality what you can do very, you know, every person who's 852 00:44:15,715 --> 00:44:19,986 coded for a while knows that there must be a Python library 853 00:44:19,986 --> 00:44:22,577 that calculates the sun position. 854 00:44:22,597 --> 00:44:26,383 This must exist 'cause the ecosystem is huge and people 855 00:44:26,383 --> 00:44:29,008 have validated this. It's a library that you can go 856 00:44:29,008 --> 00:44:31,801 and check the community. Are there lots of comments in 857 00:44:31,801 --> 00:44:33,997 the, and on the GitHub, you know, a page. 858 00:44:34,706 --> 00:44:36,734 And there were a few actually libraries that did the sun 859 00:44:36,734 --> 00:44:39,815 position, but very quickly it was okay, this one has a very 860 00:44:39,815 --> 00:44:42,888 nice interface to use and there are lots of, it's an active 861 00:44:42,888 --> 00:44:44,922 community. So, you know, in five minutes 862 00:44:44,922 --> 00:44:48,454 it's running, right? So I don't mean to say that AI 863 00:44:48,454 --> 00:44:52,080 can't help you because it can. But I also see a lot of that, 864 00:44:52,080 --> 00:44:55,060 like are we forgetting about good engineering principles? 865 00:44:55,153 --> 00:44:57,400 Are we forgetting about good customer service? 866 00:44:57,775 --> 00:45:00,535 Can we understand what the most common queries are? 867 00:45:00,535 --> 00:45:02,965 Created a frequently asked question section. 868 00:45:03,025 --> 00:45:04,885 Send emails to remind people of stuff. 869 00:45:05,209 --> 00:45:08,696 Cause a lot of your problems can be fixed by targeting the 870 00:45:08,696 --> 00:45:10,546 underlying causes of the issue, not by using AI. 871 00:45:12,143 --> 00:45:15,194 Yeah, I was laughing when you explained that, because I think 872 00:45:15,194 --> 00:45:17,114 it's such a quite insightful, right? 873 00:45:17,114 --> 00:45:20,853 So because people think just by applying AI, AI can do smart 874 00:45:20,853 --> 00:45:23,873 things that can, you know, solve all the problems, you know, 875 00:45:23,873 --> 00:45:26,332 seemingly, right? But I think we know, like it's 876 00:45:26,332 --> 00:45:28,140 probably not possible to get a hundred percent. 877 00:45:28,140 --> 00:45:31,713 And I was even frustrated sometimes if I get response from 878 00:45:31,713 --> 00:45:36,093 like seemingly an AI, right? Uh, that tries to even first 879 00:45:36,093 --> 00:45:40,048 re-explain what I just express as my complaint or, you know, 880 00:45:40,048 --> 00:45:42,222 problem. And then it gives like some 881 00:45:42,222 --> 00:45:44,976 basic information that I should have known in the first place, 882 00:45:44,976 --> 00:45:46,476 right? And then I have to wait again. 883 00:45:46,896 --> 00:45:48,726 So I think it's always very frustrating. 884 00:45:49,199 --> 00:45:52,843 But these days people, when they build AI products, they seem to 885 00:45:52,843 --> 00:45:57,425 always rely on this thing called eval method or eval test, to 886 00:45:57,425 --> 00:46:01,534 actually, you know, kind of like train your solutions such that 887 00:46:01,534 --> 00:46:03,656 it doesn't deviate so much probabilistically. 888 00:46:03,896 --> 00:46:07,706 So tell us about this approach. Is it something that is useful 889 00:46:07,706 --> 00:46:11,784 and people have to rely on? Or is it something also that 890 00:46:11,784 --> 00:46:15,596 feels, uh, you know, like impossible to actually come up 891 00:46:15,596 --> 00:46:19,567 with 100% accurate solution? Um, I don't know exactly that 892 00:46:19,567 --> 00:46:23,046 approach, what it is, but I think I've been saying to people 893 00:46:23,046 --> 00:46:25,372 for a long time, you need to, whenever you're going to build 894 00:46:25,372 --> 00:46:29,434 an AI-based product, you need to create your own benchmark to 895 00:46:29,434 --> 00:46:33,990 measure the performance of AI at the task you're trying to solve. 896 00:46:35,190 --> 00:46:37,677 There's a whole process to do that, that is a little bit more 897 00:46:37,677 --> 00:46:40,574 complicated than it seems, because you do need a validation 898 00:46:40,574 --> 00:46:43,230 set which is separate from a test set. 899 00:46:43,260 --> 00:46:47,478 Your validation set you use, you use it to select among different 900 00:46:47,478 --> 00:46:50,430 models, among different options to pick the best one. 901 00:46:51,115 --> 00:46:53,925 That's a little bit biased 'cause you have selection bias, 902 00:46:53,925 --> 00:46:55,430 right? You could pick something that 903 00:46:55,430 --> 00:46:57,385 works really well only on that validation set. 904 00:46:57,385 --> 00:47:00,655 And that's why you need the test set at the end where you perform 905 00:47:00,655 --> 00:47:03,025 an additional check, right? And you never go back. 906 00:47:03,025 --> 00:47:05,143 If you don't like the result of the test set, you need to 907 00:47:05,143 --> 00:47:06,715 collect a new test set with new data. 908 00:47:07,197 --> 00:47:09,339 A lot of people don't do this carefully enough and you know, 909 00:47:09,339 --> 00:47:12,717 it's up to you to see how far you want to go to that process. 910 00:47:12,717 --> 00:47:15,570 But I've been telling people you do need a benchmark to start 911 00:47:15,570 --> 00:47:17,507 with. And then once you have your 912 00:47:17,507 --> 00:47:19,737 benchmark, you can see if a RAG approach works better. 913 00:47:19,737 --> 00:47:22,979 If a fine tuning approach works better, you can compare 914 00:47:22,979 --> 00:47:24,657 different models. There's also something pretty 915 00:47:24,657 --> 00:47:28,210 catastrophic that happened a few weeks ago, which is that when 916 00:47:28,210 --> 00:47:30,891 OpenAI launched their GPT-5, is it five? 917 00:47:31,251 --> 00:47:31,971 The latest one? Yeah. 918 00:47:32,736 --> 00:47:34,866 They removed all the previous ones, right? 919 00:47:35,239 --> 00:47:37,506 And a lot of people were relying on previous models. 920 00:47:37,566 --> 00:47:41,802 So if you're forced into a new model, then you need the way of 921 00:47:41,892 --> 00:47:44,576 benchmark, benchmarking to understand if the performance 922 00:47:44,576 --> 00:47:47,074 has changed on the problem you're solving. 923 00:47:47,694 --> 00:47:52,708 So yes, now this will tell you how good AI is for the job. 924 00:47:52,708 --> 00:47:55,408 It may not be good enough or may be good enough, you know. 925 00:47:55,860 --> 00:47:58,530 It will help you assess, you know, the hallucinations that... 926 00:47:59,437 --> 00:48:02,317 I would also be careful with just counting errors, you know. 927 00:48:02,407 --> 00:48:06,037 You also may want to qualify them because sometimes just by 928 00:48:06,037 --> 00:48:09,337 counting them, you don't realize how bad the errors can be and 929 00:48:09,337 --> 00:48:12,647 the impact they can have. You know, people have been 930 00:48:12,647 --> 00:48:17,012 saying a surgeon that is 99%, has a 99% survival rate versus 931 00:48:17,012 --> 00:48:20,060 an AI surgeon that has 99.5, which one would you pick and 932 00:48:20,060 --> 00:48:22,618 stuff? Okay, but does the AI surgeon 933 00:48:22,618 --> 00:48:26,388 cut the wrong leg sometimes, even if he doesn't do it very 934 00:48:26,388 --> 00:48:28,789 often, you know. That's a kind of a question. 935 00:48:29,473 --> 00:48:31,969 But yeah, I think it's a good approach to do the benchmarking 936 00:48:31,969 --> 00:48:34,676 to, you know, try to understand, do hallucinations matter, 937 00:48:34,676 --> 00:48:37,301 because they probably will still be there. 938 00:48:37,301 --> 00:48:40,690 Do errors matter? So do we still need a human into 939 00:48:40,690 --> 00:48:43,952 the loop, do we not? Lots of things like that, that I 940 00:48:43,952 --> 00:48:45,969 would definitely, is there another solution, like I said 941 00:48:45,969 --> 00:48:47,619 with customer service. Can we do something else? 942 00:48:47,619 --> 00:48:51,239 What's the real problem? Another question is, is the 943 00:48:51,239 --> 00:48:53,863 problem I'm trying to solve a conversational problem? 944 00:48:54,163 --> 00:48:57,862 Is there language involved? Because I see people do this, 945 00:48:57,862 --> 00:49:01,023 they're building a travel planner where you actually input 946 00:49:01,023 --> 00:49:04,251 destinations from a box. You select them from a box, like 947 00:49:04,251 --> 00:49:06,259 a it, not a box, like a dropdown, right? 948 00:49:06,818 --> 00:49:10,860 Or a search box. And then what it does is suggest 949 00:49:10,860 --> 00:49:12,981 an itinerary in a very structured format. 950 00:49:13,776 --> 00:49:17,175 Under the hood, this company was trying to solve it by using an 951 00:49:17,175 --> 00:49:19,714 LLM. So they would take all the 952 00:49:19,714 --> 00:49:23,236 user's query, transform it into a natural language problem, give 953 00:49:23,236 --> 00:49:25,922 it to the LLM, then the LLM returns something, you would 954 00:49:25,922 --> 00:49:28,206 have to parse that result to convert it into. 955 00:49:28,819 --> 00:49:30,846 And it was super slow, for starters, right? 956 00:49:30,846 --> 00:49:34,475 It was really, really slow. And I told this company, nothing 957 00:49:34,475 --> 00:49:36,821 of this has anything to do with language. 958 00:49:37,991 --> 00:49:40,461 And the easier solution that they actually found was we're 959 00:49:40,461 --> 00:49:44,183 just gonna use a Google Places API, which just gives you, you 960 00:49:44,183 --> 00:49:47,426 know, 'cause it was an itinerary planning to visit a place, 961 00:49:47,426 --> 00:49:48,821 right? Or a city. 962 00:49:48,821 --> 00:49:52,646 So you can actually get up-to-date opening hours, which 963 00:49:52,646 --> 00:49:55,889 ChatGPT can't do. So you just go and get them from 964 00:49:55,889 --> 00:49:58,742 Google Places and you use a typical algorithm for like 965 00:49:58,742 --> 00:50:01,503 pathfinding, you know, like Dijkstra's algorithm, which is 966 00:50:01,503 --> 00:50:04,069 there are libraries for that. So they kind of and that was 967 00:50:04,069 --> 00:50:06,085 really fast. And they solve a problem without 968 00:50:06,085 --> 00:50:09,511 any LLM. So LLMs are best used when 969 00:50:09,511 --> 00:50:12,939 actually for a chatbot, for instance, when it involves 970 00:50:12,939 --> 00:50:14,969 language. But a lot of people just trying 971 00:50:14,969 --> 00:50:18,004 to use them for everything. Yeah, I think that definitely, 972 00:50:18,004 --> 00:50:20,362 it's kind of like people think it's a holy grail, right? 973 00:50:20,362 --> 00:50:22,755 And in fact, I think sometimes maybe because of the hypes, 974 00:50:22,755 --> 00:50:24,664 right? People just creating new 975 00:50:24,664 --> 00:50:28,790 libraries, new tools, new stuff that works, that relies on LLM 976 00:50:28,790 --> 00:50:30,832 or maybe prompts, so to speak, right? 977 00:50:30,832 --> 00:50:33,860 They kind of like fit in, you know, the solution with these 978 00:50:33,860 --> 00:50:35,392 prompt engineering kind of thing, right? 979 00:50:35,392 --> 00:50:38,482 Try to come up with a, you know, as best prompt as possible, give 980 00:50:38,482 --> 00:50:41,332 as much context as possible. But you also mentioned, you 981 00:50:41,332 --> 00:50:43,162 highlight when the foundational model change. 982 00:50:43,342 --> 00:50:45,242 Definitely, things probabilistically will change, 983 00:50:45,242 --> 00:50:48,502 maybe sometimes by a lot, maybe sometimes not by a lot. 984 00:50:48,772 --> 00:50:51,190 But still you need to kind of like validate and test your 985 00:50:51,190 --> 00:50:52,980 approach, right? Because when this happens, 986 00:50:52,980 --> 00:50:55,582 probabilistically, you might give a wrong output and 987 00:50:55,582 --> 00:50:59,684 suggestion as well. So I wanna move on to the next 988 00:50:59,684 --> 00:51:02,617 section by, you know, discussing about the concerns people have. 989 00:51:02,677 --> 00:51:05,221 Uh, one thing about AI, it creates a lot of possibility, 990 00:51:05,221 --> 00:51:08,071 but there are also a lot of concerns from people that, you 991 00:51:08,071 --> 00:51:10,997 know, what will happen to my job, what will happen to my 992 00:51:10,997 --> 00:51:13,944 role, what will happen to, you know, the world economy and all 993 00:51:13,944 --> 00:51:16,384 that. So maybe from the first 994 00:51:16,384 --> 00:51:18,869 discussion, I wanna highlight the concerns, right? 995 00:51:19,079 --> 00:51:21,825 Uh, especially for tech, software engineers think there 996 00:51:21,825 --> 00:51:25,311 are many, many software engineers that can be replaced, 997 00:51:25,311 --> 00:51:27,851 including juniors. What is your opinion about this? 998 00:51:28,297 --> 00:51:30,127 Yeah. Well, I-I've written a lot about 999 00:51:30,127 --> 00:51:33,307 this in my second book on AI, which is called the AI Pocket 1000 00:51:33,307 --> 00:51:34,952 Book, which we haven't spoken about yet. 1001 00:51:35,315 --> 00:51:38,655 But part of the goal of that book was to answer these 1002 00:51:38,655 --> 00:51:41,225 questions. I was approached by a publisher. 1003 00:51:41,745 --> 00:51:45,892 They thought, we need a book on surviving AI for software 1004 00:51:45,892 --> 00:51:48,537 developers, right? And I was like, okay, I will 1005 00:51:48,537 --> 00:51:51,905 definitely speak about the whole job market situation. 1006 00:51:52,521 --> 00:51:56,551 I think that there will be jobs that will be lost to AI for 1007 00:51:56,551 --> 00:51:58,528 sure, maybe prematurely sometimes. 1008 00:51:58,528 --> 00:52:01,030 'Cause we've already seen companies firing people and now 1009 00:52:01,030 --> 00:52:03,108 hiring them back because it didn't work. 1010 00:52:04,089 --> 00:52:07,203 So I think that, I mean, this is, this could be a, you know, 1011 00:52:07,203 --> 00:52:10,317 we can speak about this from many angles, but for the angle 1012 00:52:10,317 --> 00:52:14,043 of a person who wants to preserve their job, I would say 1013 00:52:14,043 --> 00:52:16,449 that there are different kinds of jobs. 1014 00:52:16,929 --> 00:52:21,789 Some jobs, AI may be able to replace, and some, it won't. 1015 00:52:21,789 --> 00:52:25,154 So as a professional, you want to position yourself in that 1016 00:52:25,154 --> 00:52:28,586 latter category, right? I think that there's a certain 1017 00:52:28,586 --> 00:52:31,236 level of work that requires a lot of excellence. 1018 00:52:32,467 --> 00:52:35,782 When you go above and beyond, when you rely on a network of 1019 00:52:35,782 --> 00:52:38,636 other people, you need interactions, you need multiple 1020 00:52:38,636 --> 00:52:41,050 skills. In the business world, that job 1021 00:52:41,050 --> 00:52:43,073 is oft-... In the software world, it's 1022 00:52:43,073 --> 00:52:45,268 often at the intersection between software and business. 1023 00:52:45,568 --> 00:52:50,388 So you are not a person who takes a specification of a UI 1024 00:52:50,388 --> 00:52:54,475 and codes that specification to the letter without an opinion, 1025 00:52:54,475 --> 00:52:57,885 right? You are the person who will help 1026 00:52:57,885 --> 00:53:00,448 the business choose the best feature to build. 1027 00:53:01,104 --> 00:53:02,837 There are lots of positions that are like that. 1028 00:53:02,897 --> 00:53:05,784 If you work for startups, for example, you may need to be a 1029 00:53:05,784 --> 00:53:08,472 jack of all trades, a person helping select features. 1030 00:53:09,003 --> 00:53:12,151 As you progress in your career, you're closer to the business 1031 00:53:12,151 --> 00:53:15,439 very often. So my advice is often try to 1032 00:53:15,439 --> 00:53:18,331 work at the intersection between business and technology. 1033 00:53:18,361 --> 00:53:22,339 See yourself as a person who helps build commercially 1034 00:53:22,339 --> 00:53:24,826 successful products. Not just write software. 1035 00:53:25,556 --> 00:53:28,682 Because I think you will be quite irreplaceable when you're 1036 00:53:28,682 --> 00:53:32,216 that person. And another thing I think is 1037 00:53:32,216 --> 00:53:35,606 that a lot of people speak about other people's jobs. 1038 00:53:35,606 --> 00:53:38,590 They say these people will lose their job, but they've never met 1039 00:53:38,590 --> 00:53:41,990 a person who does that job and they don't know what it takes to 1040 00:53:41,990 --> 00:53:44,636 do the job, right? Especially at the excellent 1041 00:53:44,636 --> 00:53:47,174 level, right? I've seen this a lot. 1042 00:53:47,294 --> 00:53:49,926 You know, people say screenwriters will get replaced 1043 00:53:49,926 --> 00:53:54,194 with AI or replaced by AI. But then, have they met a 1044 00:53:54,194 --> 00:53:56,403 screenwriter? Have they seen what it actually 1045 00:53:56,403 --> 00:53:59,985 takes to produce a show that people love, that people laugh 1046 00:53:59,985 --> 00:54:06,205 at or laugh with, you know? I am listening to this podcast 1047 00:54:06,205 --> 00:54:10,241 called Office Ladies, which is, by the actresses from The Office 1048 00:54:10,241 --> 00:54:13,553 where they tell you stories behind the show and you can see 1049 00:54:13,553 --> 00:54:16,615 how much thinking there was sometimes about certain 1050 00:54:16,615 --> 00:54:20,181 storylines, you know, that they decided to include in the show. 1051 00:54:20,247 --> 00:54:23,397 There's so much that goes on, you know, in writing a show. 1052 00:54:23,547 --> 00:54:27,595 Yes, there are some cookie cutter show, soap opera kind of 1053 00:54:27,595 --> 00:54:31,246 show that maybe AI can do, but there's so much more that 1054 00:54:31,246 --> 00:54:33,150 happens when people are writing a show. 1055 00:54:33,150 --> 00:54:34,770 And the same thing with programming with code. 1056 00:54:35,010 --> 00:54:39,237 I feel like maybe I'm also a bit more senior in my career, but 1057 00:54:39,237 --> 00:54:44,500 most of my work in software is not writing code in terms of, 1058 00:54:44,500 --> 00:54:48,570 you know, I'm gonna move this button from here to there. 1059 00:54:48,570 --> 00:54:52,560 There's a lot of thinking about security, about architecture. 1060 00:54:52,800 --> 00:54:54,950 A lot of this has to do with negotiating with the business. 1061 00:54:55,700 --> 00:54:58,362 A lot of it, you know, the business wants to do this, but 1062 00:54:58,362 --> 00:55:00,630 this is really big and they don't know it. 1063 00:55:00,750 --> 00:55:04,000 So I explain to them why it's big and then I propose a 1064 00:55:04,000 --> 00:55:05,820 different solution. What if we do this other thing? 1065 00:55:05,820 --> 00:55:09,900 It's not as great, but we can do it quickly and then we can see 1066 00:55:09,900 --> 00:55:12,028 how people react. And then we see if we build the 1067 00:55:12,028 --> 00:55:14,658 bigger thing that you want. All of these help, you know, as 1068 00:55:14,658 --> 00:55:17,183 part of software development, but it's not writing lines of 1069 00:55:17,183 --> 00:55:20,564 code. So if I had to recommend to a 1070 00:55:20,564 --> 00:55:23,563 software developer, again, I would say, a lot of interaction 1071 00:55:23,563 --> 00:55:26,984 with business, a lot of high level work business, learn about 1072 00:55:26,984 --> 00:55:28,874 business, you know, read business books. 1073 00:55:29,024 --> 00:55:31,949 'Cause a lot of us, we have no idea about anything related to 1074 00:55:31,949 --> 00:55:34,644 business, to sales, to value propositions. 1075 00:55:34,644 --> 00:55:37,194 How to define a value proposition, for example. 1076 00:55:37,542 --> 00:55:40,453 There's a book called Value Proposition Design, which I, 1077 00:55:40,453 --> 00:55:42,174 which I recommend. So yeah. 1078 00:55:42,174 --> 00:55:46,949 I think that a good way to protect that your job is that, 1079 00:55:46,949 --> 00:55:49,233 you know, yeah. Yeah. 1080 00:55:49,233 --> 00:55:51,274 So I think that's very good advice, right? 1081 00:55:51,274 --> 00:55:55,137 Because obviously when we talk about AI replacing jobs is 1082 00:55:55,137 --> 00:55:57,529 probably the first, will be the mundane things. 1083 00:55:57,559 --> 00:56:00,238 You know, like producing lines of code, translating from, you 1084 00:56:00,238 --> 00:56:03,692 know, one thing to another, or even like maybe when you say 1085 00:56:03,692 --> 00:56:06,101 about screenwriter, maybe not necessarily screenwriter, but 1086 00:56:06,101 --> 00:56:09,695 someone who polish, you know, some writings that, some drafts 1087 00:56:09,695 --> 00:56:12,233 that people do, right? So those kind of things, 1088 00:56:12,233 --> 00:56:14,611 definitely the first prospect that could be replaced, right? 1089 00:56:14,911 --> 00:56:17,161 But if we keep moving to the higher level. 1090 00:56:17,161 --> 00:56:19,066 Like when we talk about programming language, also the 1091 00:56:19,066 --> 00:56:21,571 same, right, it started with like machine language then 1092 00:56:21,571 --> 00:56:22,819 becomes general programming language. 1093 00:56:22,819 --> 00:56:24,649 Now you have conversational language. 1094 00:56:24,649 --> 00:56:27,751 If we can think of ourselves moving to the higher abstraction 1095 00:56:27,751 --> 00:56:31,711 maybe, so to speak, maybe we can even amplify, you know, the 1096 00:56:31,711 --> 00:56:33,587 value that we can produce, right? 1097 00:56:33,677 --> 00:56:37,005 So I think I agree what you said that software engineers need to 1098 00:56:37,005 --> 00:56:40,129 understand more about business or maybe thinking in terms of, 1099 00:56:40,129 --> 00:56:43,657 you know, how can we make this more successful, commercial, 1100 00:56:43,657 --> 00:56:47,261 some psychological aspect, human aspect that probably software 1101 00:56:47,261 --> 00:56:51,077 engineers can also explore. So how about the juniors here? 1102 00:56:51,107 --> 00:56:53,833 Because I think the juniors might be kind of like 1103 00:56:53,833 --> 00:56:56,162 frustrated, right? So I just finished my study for 1104 00:56:56,162 --> 00:56:58,320 example. But now I'm finding it hard to 1105 00:56:58,320 --> 00:57:00,354 find a job. So do you have any advice for 1106 00:57:00,354 --> 00:57:02,595 them? I mean I spoke about this 1107 00:57:02,595 --> 00:57:04,806 excellence side of things, but there are other things that will 1108 00:57:04,806 --> 00:57:08,512 be helpful. One of them is when you work on 1109 00:57:08,512 --> 00:57:13,365 very custom, customized work if you want, that is very kind of 1110 00:57:13,365 --> 00:57:16,449 complicated and narrowly specified and stuff. 1111 00:57:16,966 --> 00:57:19,480 If you work on complicated stuff, and I can give you a 1112 00:57:19,480 --> 00:57:21,988 couple examples, those things are unlikely to be replaced by 1113 00:57:21,988 --> 00:57:25,360 AI, because first of all, you can't even describe to an AI 1114 00:57:25,360 --> 00:57:28,586 what you need to do. Even the business people will 1115 00:57:28,586 --> 00:57:30,496 not understand. And, uh, I can give you an 1116 00:57:30,496 --> 00:57:33,581 example. I worked on a project that 1117 00:57:33,581 --> 00:57:37,403 involved modeling the propagation of temperature or of 1118 00:57:37,403 --> 00:57:40,966 heat inside a building, right? And I was writing the software 1119 00:57:40,966 --> 00:57:43,237 for this, right? All these complicated 1120 00:57:43,237 --> 00:57:47,275 calculations, but it was so custom and they were devices 1121 00:57:47,275 --> 00:57:51,094 connected to buildings using APIs where we get the data and 1122 00:57:51,094 --> 00:57:53,551 then physical models. And there were parameters that 1123 00:57:53,551 --> 00:57:55,815 define, you know, the insulation of the building. 1124 00:57:56,505 --> 00:57:59,875 It's very hard to, for me to even imagine how you could 1125 00:57:59,875 --> 00:58:04,434 describe all of that in a prompt which is not the same as writing 1126 00:58:04,434 --> 00:58:08,876 the code essentially, right? Uh, so I would say you want to 1127 00:58:08,876 --> 00:58:12,655 specialize or to gain experience on tightly controlled kind of 1128 00:58:12,655 --> 00:58:15,575 software. An example of what's not like 1129 00:58:15,575 --> 00:58:19,424 that is, you know, building an prototype just to show something 1130 00:58:19,424 --> 00:58:22,080 to a potential client, but it doesn't really work. 1131 00:58:22,372 --> 00:58:25,672 There's a lot of MVP or no-code kind of thing going on there. 1132 00:58:25,672 --> 00:58:28,230 It's like we're gonna build not a really functional thing that 1133 00:58:28,230 --> 00:58:30,631 works with lots of clients, with lots of customer requirements, 1134 00:58:30,631 --> 00:58:34,054 we're just gonna build a very rough prototype to validate an 1135 00:58:34,054 --> 00:58:37,173 idea, right? I think those are the jobs where 1136 00:58:37,173 --> 00:58:40,104 people are gonna be replacing that with AI a lot. 1137 00:58:40,764 --> 00:58:44,430 But then you can do very, very custom kind of like, focus on 1138 00:58:44,430 --> 00:58:47,094 really tightly defined, custom complicated problems. 1139 00:58:47,094 --> 00:58:50,286 You know, if they can be a bit scientific or they require 1140 00:58:50,286 --> 00:58:53,049 mathematics, you know, it's likely business people will not 1141 00:58:53,049 --> 00:58:54,684 have studied those things either. 1142 00:58:54,804 --> 00:58:57,712 So they won't be able to know where to start with those 1143 00:58:57,712 --> 00:59:01,287 problems. Also working on anything that is 1144 00:59:01,287 --> 00:59:05,373 mission critical will help you. When I say mission critical, 1145 00:59:05,373 --> 00:59:08,537 something that you deploy a piece of software that could 1146 00:59:08,537 --> 00:59:14,193 lose a company a lot of money if it wasn't used correctly or it 1147 00:59:14,193 --> 00:59:19,605 could even cause harm to people. Because this is going to require 1148 00:59:19,605 --> 00:59:24,701 a lot of manual validation of manual input of, you know, I 1149 00:59:24,701 --> 00:59:27,935 don't think that's going to be replaced by AI anytime soon. 1150 00:59:27,935 --> 00:59:30,275 So if you can become the person who learns about those things, 1151 00:59:30,275 --> 00:59:33,899 then you'll be better placed to find a job or to preserve your 1152 00:59:33,899 --> 00:59:36,452 job. So I hope people could learn 1153 00:59:36,452 --> 00:59:38,645 from these tips. So when you mentioned about 1154 00:59:38,645 --> 00:59:41,133 building prototypes, right? My mind straight, straightly go 1155 00:59:41,133 --> 00:59:44,210 to vibe coding, right? Uh, I think many people think, 1156 00:59:44,210 --> 00:59:47,642 you know, now everyone can code, you know, just doing vibe coding 1157 00:59:47,642 --> 00:59:51,153 and even it might be creating a perception that you can be a 1158 00:59:51,153 --> 00:59:53,735 vibe coder, you know. There might be a job role as a 1159 00:59:53,735 --> 00:59:55,097 vibe coder specializing in vibe coder. 1160 00:59:55,367 --> 00:59:56,717 Same thing like prompt engineering. 1161 00:59:56,717 --> 00:59:59,813 You know, I can be a prompt engineer that can solve, you 1162 00:59:59,813 --> 01:00:03,077 know, maybe general purpose AI conversational kind of a thing. 1163 01:00:03,467 --> 01:00:07,645 So maybe tell us, uh, your view about this and, and will in the 1164 01:00:07,645 --> 01:00:10,546 future, we see new job types being created, something like 1165 01:00:10,546 --> 01:00:12,889 this, or is it more like a hype now? 1166 01:00:13,180 --> 01:00:16,579 Now I think there is a place for everything and there is a place 1167 01:00:16,579 --> 01:00:20,303 for a job title that is a vibe coder potentially, because we've 1168 01:00:20,303 --> 01:00:23,635 seen that with Bubble, you know, Bubble, the no-code tool. 1169 01:00:23,875 --> 01:00:26,357 I've seen people who specialize because you need to actually 1170 01:00:26,357 --> 01:00:28,843 learn how to use those tools at the end of the day, especially 1171 01:00:28,843 --> 01:00:30,595 if you want to build something custom. 1172 01:00:30,835 --> 01:00:33,055 Which is kind of part of the problem here, is if you just 1173 01:00:33,055 --> 01:00:36,705 type it a little prompt and you accept what comes out of it, 1174 01:00:36,705 --> 01:00:39,255 then it's fine. But if you want to start, you 1175 01:00:39,255 --> 01:00:41,849 know, like with no-code, you often need to learn what a 1176 01:00:41,849 --> 01:00:44,993 platform does and how to use it, how to connect it to data 1177 01:00:44,993 --> 01:00:47,575 sources and all of that. And I've seen people who 1178 01:00:47,575 --> 01:00:50,345 specialize in helping companies build with Bubble, right? 1179 01:00:50,345 --> 01:00:52,685 Because it's cheaper, it's quicker, right? 1180 01:00:53,075 --> 01:00:55,535 You're not gonna build production-grade software. 1181 01:00:55,565 --> 01:00:57,485 That's the illusion that we need to kind of eliminate. 1182 01:00:57,515 --> 01:01:00,741 And whenever you want something really custom, it's as difficult 1183 01:01:00,741 --> 01:01:02,405 as coding. Because coding is that. 1184 01:01:02,405 --> 01:01:05,210 It's literally telling the machine what you want it to do, 1185 01:01:05,210 --> 01:01:07,973 right? So the illusion that, that's the 1186 01:01:07,973 --> 01:01:10,137 same as building production-grade serious 1187 01:01:10,137 --> 01:01:12,892 software, I think that illusion needs to go. 1188 01:01:13,312 --> 01:01:17,040 But there's definitely room for that other thing in between 1189 01:01:17,040 --> 01:01:21,600 where you help with MVPs, where you help with prototypes, right? 1190 01:01:21,941 --> 01:01:27,976 We could see UI designers maybe using AI to create better 1191 01:01:27,976 --> 01:01:31,363 prototypes for their clients, which are more functional than 1192 01:01:31,363 --> 01:01:33,542 before. So there will be changes 1193 01:01:33,542 --> 01:01:35,865 definitely in there. And there will be new jobs and 1194 01:01:35,865 --> 01:01:37,053 new opportunities, I think. Yeah. 1195 01:01:37,903 --> 01:01:41,422 Yeah, so hopefully as with a new, I dunno, technological 1196 01:01:41,422 --> 01:01:44,494 disruption, I think there will be a lot of fears in the very 1197 01:01:44,494 --> 01:01:46,303 beginning. But hopefully after that we can 1198 01:01:46,303 --> 01:01:49,357 like thrive and, you know, maybe in the new opportunities people 1199 01:01:49,357 --> 01:01:53,068 move on to new kind of skillset and solving bigger problems and 1200 01:01:53,068 --> 01:01:55,286 things like that. So hopefully that will be the 1201 01:01:55,286 --> 01:01:57,253 case. And speaking about that, right? 1202 01:01:57,253 --> 01:02:00,221 So I wanna touch on this one thing that I think it's, uh, 1203 01:02:00,221 --> 01:02:03,701 worth to discuss shortly, right? So the AGI possibility. 1204 01:02:03,761 --> 01:02:08,318 So I think many AI companies think AGI will happen, I dunno, 1205 01:02:08,318 --> 01:02:10,107 three years, five years, whatever that is. 1206 01:02:10,317 --> 01:02:12,687 Sometimes I listen to podcasts as well, you know. 1207 01:02:12,897 --> 01:02:17,125 There are seemingly AI experts telling that humanity is in 1208 01:02:17,125 --> 01:02:20,429 crisis. So personally do you think AGI 1209 01:02:20,429 --> 01:02:23,469 will happen or maybe some resemblance of it? 1210 01:02:23,469 --> 01:02:25,919 So yeah, maybe a little bit on this part. 1211 01:02:27,369 --> 01:02:30,639 Look, the first thing, okay, AGI, artificial general 1212 01:02:30,639 --> 01:02:33,021 intelligence, right? A computer that can do anything 1213 01:02:33,021 --> 01:02:35,937 a human can do. It's not very well defined, but 1214 01:02:35,937 --> 01:02:38,115 we're talking about really super powerful AI. 1215 01:02:38,500 --> 01:02:42,070 Well, the first thing I want to say, it sells so much to make 1216 01:02:42,070 --> 01:02:44,350 those predictions. There's nothing that sells more. 1217 01:02:44,565 --> 01:02:48,555 People have built entire careers around predicting that. 1218 01:02:48,555 --> 01:02:52,460 You know, uh, an example is Ray Kurzweil who's written these 1219 01:02:52,460 --> 01:02:56,054 books on the singularity, saying the singularity is going to 1220 01:02:56,054 --> 01:02:58,293 happen. Singularity is when the machines 1221 01:02:58,293 --> 01:03:02,002 become so smart that they can improve themselves at an 1222 01:03:02,002 --> 01:03:04,167 exponential rate and everything collapses. 1223 01:03:04,167 --> 01:03:07,904 And it's like like sci-fi world. It's, people are fascinated by 1224 01:03:07,904 --> 01:03:12,084 that and it sells so much. I feel like people who speak the 1225 01:03:12,084 --> 01:03:16,428 way I speak about AI, and I've heard the same experience, you 1226 01:03:16,428 --> 01:03:20,682 know, from other people, we actually struggle more to be 1227 01:03:20,682 --> 01:03:24,712 heard because people don't invite you to give talks as 1228 01:03:24,712 --> 01:03:27,209 much. Who do you want on the stage? 1229 01:03:27,239 --> 01:03:30,019 There's a reasonable person telling you AI is smart, but 1230 01:03:30,019 --> 01:03:33,210 it's also kind of dumb. Or do you want a person telling 1231 01:03:33,210 --> 01:03:35,439 you how it's going to be the Terminator, you know? 1232 01:03:35,439 --> 01:03:39,070 And it seems like the latter is, it sells a lot. 1233 01:03:39,070 --> 01:03:41,937 And when I was looking for a publisher for my first book on 1234 01:03:41,937 --> 01:03:45,357 AI, I remember I spoke with an agent that he's a pretty 1235 01:03:45,357 --> 01:03:47,407 prominent agent. Like a book agent. 1236 01:03:47,527 --> 01:03:54,067 And he told me, no, like your book won't sell because you have 1237 01:03:54,067 --> 01:03:57,501 to either say that AI will be so amazing, that will change 1238 01:03:57,501 --> 01:03:59,197 everything, or they will destroy everything. 1239 01:03:59,617 --> 01:04:03,813 If you say something kind of in the middle, nobody really like 1240 01:04:03,813 --> 01:04:06,472 that. So the first thing is like this 1241 01:04:06,472 --> 01:04:09,718 AGI thing is, and it's also self-serving because a lot of 1242 01:04:09,718 --> 01:04:12,938 people are making money from selling the shovels or selling 1243 01:04:12,938 --> 01:04:17,048 the AI to build AI or AI for the sake-, and then they tell you 1244 01:04:17,048 --> 01:04:19,821 AGI is coming, right? I would tell you the answer is 1245 01:04:19,821 --> 01:04:21,864 no. So AGI is not coming anytime 1246 01:04:21,864 --> 01:04:26,016 soon. And the reason is that the 1247 01:04:26,016 --> 01:04:30,496 current methodology used to build AI, which is machine 1248 01:04:30,496 --> 01:04:34,078 learning, has some flaws. And these flaws cause the 1249 01:04:34,078 --> 01:04:36,886 hallucinations, for example, which will prevent AGI, right? 1250 01:04:37,707 --> 01:04:42,217 They are not solvable right now within the methodology that we 1251 01:04:42,217 --> 01:04:45,064 know. This means that if a new 1252 01:04:45,064 --> 01:04:48,145 methodology, something different from the current type of machine 1253 01:04:48,145 --> 01:04:50,504 learning were invented, maybe it could be solved. 1254 01:04:50,534 --> 01:04:52,332 I don't know. There's also a question that in 1255 01:04:52,332 --> 01:04:53,664 principle, this is possible or not. 1256 01:04:53,994 --> 01:04:58,006 But even if we put that aside, if a new methodology is 1257 01:04:58,006 --> 01:05:00,892 discovered, maybe. But nobody knows that 1258 01:05:00,892 --> 01:05:04,122 methodology now, right? We don't know it and we need to 1259 01:05:04,122 --> 01:05:05,394 acknowledge that we don't know it. 1260 01:05:05,994 --> 01:05:10,584 When GPT-5 came out - and it was a flop, a big flop, and a big 1261 01:05:10,584 --> 01:05:12,904 disaster - cause it was nothing new. 1262 01:05:12,904 --> 01:05:15,324 It was maybe a little bit better, but not really. 1263 01:05:16,134 --> 01:05:19,710 I wasn't surprised because nobody has discovered that 1264 01:05:19,710 --> 01:05:23,688 secret ingredient or if you want that new methodology to solve 1265 01:05:23,688 --> 01:05:27,549 hallucinations and stuff. So to me, it was no surprise 1266 01:05:27,549 --> 01:05:30,417 that GPT-5 still hallucinated because they haven't discovered 1267 01:05:30,417 --> 01:05:33,304 anything that I know of in between, right? 1268 01:05:33,914 --> 01:05:37,164 And you can't predict scientific discoveries like that. 1269 01:05:37,554 --> 01:05:39,954 You know, the transformer architecture was the discovery 1270 01:05:39,954 --> 01:05:44,634 that led to LLMs or like to the current powerful LLMs. What's 1271 01:05:44,634 --> 01:05:45,909 the next discovery? We don't know. 1272 01:05:45,909 --> 01:05:47,299 We don't know when it's gonna happen. 1273 01:05:47,788 --> 01:05:51,819 I always say the example of nuclear fusion based power, 1274 01:05:51,819 --> 01:05:54,768 they've been telling us for decades that that's the future. 1275 01:05:54,768 --> 01:05:58,448 It's around the corner. The problem is that the 1276 01:05:58,448 --> 01:06:01,574 methodology hasn't been found, hasn't been discovered to do 1277 01:06:01,574 --> 01:06:04,428 that. And they can keep putting money 1278 01:06:04,428 --> 01:06:08,253 into it and maybe that will help, but we don't know if it's 1279 01:06:08,253 --> 01:06:11,080 gonna happen or when, right? And that's the situation with 1280 01:06:11,080 --> 01:06:14,314 AGI, right? I feel like a lot of the 1281 01:06:14,314 --> 01:06:16,487 philosophical questions around AGI, they're valid and it's good 1282 01:06:16,487 --> 01:06:18,189 to ask them. What would happen? 1283 01:06:18,299 --> 01:06:22,309 Like what happens legally if... Does a robot have feelings? 1284 01:06:22,649 --> 01:06:24,794 I like those questions. I think they're fascinating, but 1285 01:06:24,794 --> 01:06:27,089 they are not the reality of current machine learning. 1286 01:06:27,089 --> 01:06:30,168 Like this thing of I'm going to learn from data how to predict 1287 01:06:30,168 --> 01:06:32,516 the next word. It's not going to lead us there 1288 01:06:32,516 --> 01:06:33,666 and it's gonna keep hallucinating. 1289 01:06:34,253 --> 01:06:36,503 But then if a new thing is discovered, you know, maybe. 1290 01:06:37,563 --> 01:06:39,603 Right. Pretty exciting answer, right? 1291 01:06:39,603 --> 01:06:42,436 So I think people sometimes, uh, I don't know, sometimes yeah, 1292 01:06:42,436 --> 01:06:44,982 they project something that seemingly bombastic, and then 1293 01:06:44,982 --> 01:06:46,923 yeah, it sells like what you mentioned, right? 1294 01:06:47,283 --> 01:06:50,701 But we have seen it, with the ChatGPT-5 release, right? 1295 01:06:50,701 --> 01:06:52,051 So it kind of like flop a little bit. 1296 01:06:52,321 --> 01:06:56,168 So maybe, you know, the AI hype is not as great as people sold 1297 01:06:56,168 --> 01:06:59,216 it to be, right? But obviously these next few 1298 01:06:59,216 --> 01:07:01,862 years will be very, very exciting, unpredictable at 1299 01:07:01,862 --> 01:07:04,124 times, right? Because there will always be new 1300 01:07:04,124 --> 01:07:06,810 things being invented and we just have to adapt, I guess. 1301 01:07:06,892 --> 01:07:10,419 My stance now, even now, is just, yeah, see, adapt, use them 1302 01:07:10,419 --> 01:07:13,583 as much as possible, understand what it's capable of and not 1303 01:07:13,583 --> 01:07:15,565 capable of, which is also very, very important. 1304 01:07:15,565 --> 01:07:19,570 And try to fit in you, yourself, your skillset, your value, into 1305 01:07:19,570 --> 01:07:21,930 those landscape, right? So I think probably that's like 1306 01:07:21,930 --> 01:07:24,930 the most practical thing. So Emmanuel, thank you so much 1307 01:07:24,930 --> 01:07:26,946 for this conversation. So I know we are a little bit 1308 01:07:26,946 --> 01:07:29,430 over time, but I have one last question that I have to ask you. 1309 01:07:29,550 --> 01:07:31,530 This is like a tradition in my podcast. 1310 01:07:31,740 --> 01:07:34,050 I call this the three technical leadership wisdom. 1311 01:07:34,050 --> 01:07:37,230 So if you think of them just like advice, uh, would you be 1312 01:07:37,230 --> 01:07:38,610 able to share your version today? 1313 01:07:39,516 --> 01:07:43,733 Yeah. So I would say, look, the first 1314 01:07:43,733 --> 01:07:47,108 one is that there's a term called a moat. 1315 01:07:47,228 --> 01:07:50,558 As in M-O-A-T, right? Like a protective moat. 1316 01:07:51,188 --> 01:07:53,918 It's also called a competitive advantage. 1317 01:07:54,918 --> 01:07:59,994 I think a lot of people don't understand much what this means 1318 01:07:59,994 --> 01:08:03,696 or they've read some books that tell 'em this is a competitive 1319 01:08:03,696 --> 01:08:05,642 advantage when it's actually, it's actually not. 1320 01:08:05,882 --> 01:08:07,710 I think there's a lot of confusion around the competitive 1321 01:08:07,710 --> 01:08:10,230 advantage, and I've been doing a lot of research myself on the 1322 01:08:10,230 --> 01:08:13,126 matter. What I mean with all this is 1323 01:08:13,126 --> 01:08:16,558 that you need to understand that building a successful business 1324 01:08:16,558 --> 01:08:20,176 is a business problem. It's not a technology problem. 1325 01:08:20,367 --> 01:08:23,285 It's usually, it doesn't, no business has failed because you 1326 01:08:23,285 --> 01:08:25,846 use Python 3.5 instead of, you know, whatever. 1327 01:08:26,627 --> 01:08:30,026 And the role of a business person or even the goal of a 1328 01:08:30,026 --> 01:08:32,850 business is, even if we don't like it, is to create a sort of 1329 01:08:32,850 --> 01:08:35,393 monopoly. You want a moat that protects 1330 01:08:35,393 --> 01:08:38,903 your business from competitors. You need access to some, a 1331 01:08:38,903 --> 01:08:43,153 special way to keep your clients or to gain more clients or to 1332 01:08:43,153 --> 01:08:45,127 access resources cheaper than anyone else. 1333 01:08:45,127 --> 01:08:46,926 Like there needs to be a special thing. 1334 01:08:46,926 --> 01:08:50,593 And that thing is called a moat. And one of the reasons I tell 1335 01:08:50,593 --> 01:08:54,064 you, I think technology leaders need to keep this in mind is 1336 01:08:54,064 --> 01:08:56,617 that there are a few ideas going around that are terrible. 1337 01:08:56,827 --> 01:09:01,037 One of them is this 10x thing. You build technology that is 10x 1338 01:09:01,037 --> 01:09:04,045 better than other technology and then you're done, your 1339 01:09:04,045 --> 01:09:06,037 businesses. That's not true. 1340 01:09:06,397 --> 01:09:09,205 And the proof of that ChatGPT was definitely, I think a 10x 1341 01:09:09,205 --> 01:09:11,167 kind of thing. And the business is doing 1342 01:09:11,167 --> 01:09:12,679 terribly. They're losing billions every 1343 01:09:12,679 --> 01:09:15,210 year. And the problem is they have no 1344 01:09:15,210 --> 01:09:16,567 moat. There's nothing special. 1345 01:09:16,567 --> 01:09:20,053 Having a specific type of technology doesn't build a moat 1346 01:09:20,053 --> 01:09:24,685 around your business. What is aggravating here is that 1347 01:09:24,685 --> 01:09:27,457 everybody knows how ChatGPT works, right? 1348 01:09:27,462 --> 01:09:29,511 Because it wasn't even invented by OpenAI. 1349 01:09:29,527 --> 01:09:32,569 It was invented by Google researchers who published the 1350 01:09:32,569 --> 01:09:36,180 work on a scientific paper. And I've been saying this to 1351 01:09:36,180 --> 01:09:39,729 people, right, a lot. And I kind of predicted this, I 1352 01:09:39,729 --> 01:09:43,319 wrote a book called Siliconned also where I wrote about this, 1353 01:09:43,319 --> 01:09:46,759 about OpenAI and stuff. And then what happened now is 1354 01:09:46,759 --> 01:09:50,835 Deepseek comes out. And then the world is like, oh 1355 01:09:50,835 --> 01:09:54,355 my God, it works as good as, you know, OpenAI models and what's 1356 01:09:54,355 --> 01:09:56,590 this gonna mean? Yeah, we knew this was gonna 1357 01:09:56,590 --> 01:09:59,362 happen because there's so much money to be made probably in 1358 01:09:59,362 --> 01:10:03,220 the, in AI that a lot of other entrants, competitors, copycats 1359 01:10:03,550 --> 01:10:05,080 are going to try to build the same thing. 1360 01:10:05,350 --> 01:10:08,740 So you do not actually build a successful business around 1361 01:10:08,740 --> 01:10:13,180 having superior technology. You build them around, for 1362 01:10:13,180 --> 01:10:15,400 example, network effects that keep people engaged. 1363 01:10:15,430 --> 01:10:17,949 Or you build them around switching costs. 1364 01:10:18,279 --> 01:10:21,113 People don't want to leave you because they're kind of stuck, 1365 01:10:21,113 --> 01:10:23,445 you know. With Apple, you have all Apple 1366 01:10:23,445 --> 01:10:27,011 ecosystems, so you don't want to switch to a different kind of 1367 01:10:27,011 --> 01:10:29,084 phone. Those are the things that 1368 01:10:29,084 --> 01:10:30,524 actually make or break a business. 1369 01:10:30,524 --> 01:10:33,338 It's not the technology. Um, another thing that I think 1370 01:10:33,338 --> 01:10:36,665 is very prominent in the technology world is a search 1371 01:10:36,665 --> 01:10:39,326 cost. Not a switching cost, a search 1372 01:10:39,326 --> 01:10:41,562 cost. Meaning that you want to 1373 01:10:41,562 --> 01:10:45,821 capitalize on the fact that people will find it annoying or 1374 01:10:45,821 --> 01:10:47,552 costly to search for an alternative. 1375 01:10:48,371 --> 01:10:50,831 I think Canva benefits from that, right? 1376 01:10:50,831 --> 01:10:54,831 People don't want to use anything else because they know 1377 01:10:54,831 --> 01:10:57,271 Canva and they're not gonna start searching for Canva, and 1378 01:10:57,271 --> 01:10:59,171 they don't even have opportunities to find 1379 01:10:59,171 --> 01:11:00,935 alternatives. Whenever they want to do any 1380 01:11:00,935 --> 01:11:02,801 simple graphic design, they just go to canva.com. 1381 01:11:03,572 --> 01:11:04,712 But again, why I'm telling you this? 1382 01:11:04,712 --> 01:11:08,294 Because the key to the success of this business is that the 1383 01:11:08,294 --> 01:11:13,150 product is cheap, right? If Canva tries to charge a lot 1384 01:11:13,150 --> 01:11:17,032 of money, this search cost starts to not protect the 1385 01:11:17,032 --> 01:11:19,052 business anymore because people are gonna start shopping around. 1386 01:11:19,052 --> 01:11:21,752 They're gonna, oh, Canva is now 200 a month. 1387 01:11:22,442 --> 01:11:27,007 Can I use something else? And business strategies about 1388 01:11:27,007 --> 01:11:29,471 this. And I think a lot of 1389 01:11:29,471 --> 01:11:33,245 technologists like me, we had, I didn't learn any of this in 1390 01:11:33,245 --> 01:11:34,648 university. I learned how to code. 1391 01:11:34,693 --> 01:11:37,558 I learned how to create really fancy SQL statements. 1392 01:11:38,209 --> 01:11:39,709 But that doesn't make or break a business. 1393 01:11:39,709 --> 01:11:42,259 It's business side of things and the competitive advantage is 1394 01:11:42,259 --> 01:11:46,725 super, super important. So that would be my first, uh, I 1395 01:11:46,725 --> 01:11:49,300 dunno what you call it, my leadership something. 1396 01:11:50,316 --> 01:11:54,846 Second one, there's a lot of boring stuff to be done. 1397 01:11:55,357 --> 01:11:58,661 The most successful entrepreneurs I've met have 1398 01:11:58,661 --> 01:12:02,785 built something really boring to help a boring industry do 1399 01:12:02,785 --> 01:12:04,927 something that was super inefficient, right? 1400 01:12:05,368 --> 01:12:07,655 And speaking of competitive advantage, I think one of the, 1401 01:12:07,655 --> 01:12:10,261 or the moat that they had was that it was such a niche thing 1402 01:12:10,261 --> 01:12:12,839 that it's just, it's actually a small, small market and people 1403 01:12:12,839 --> 01:12:15,877 don't want to enter the market. Once you're there, your clients 1404 01:12:15,877 --> 01:12:18,239 have switching costs that are really, you know, high. 1405 01:12:18,667 --> 01:12:22,237 And I've seen this a lot. I've seen it in aviation a lot. 1406 01:12:22,507 --> 01:12:24,757 A lot of stuff is done on paper nowadays. 1407 01:12:24,757 --> 01:12:29,608 People literally, they fill in paper forms even today to track 1408 01:12:29,608 --> 01:12:32,829 the maintenance of an aircraft or stuff like that. 1409 01:12:32,829 --> 01:12:35,919 There's so much stuff that's done badly. 1410 01:12:36,734 --> 01:12:41,454 And I think there's a lot to do in the world and in that kind of 1411 01:12:41,454 --> 01:12:45,272 boring arena. But it's not flashy, like 1412 01:12:45,272 --> 01:12:49,072 building the new goggles to pilot your drone or whatever. 1413 01:12:49,072 --> 01:12:53,569 And you know, and it's like, but actually it's a lot of fun to 1414 01:12:53,569 --> 01:12:56,512 work on things that people actually use and care about. 1415 01:12:56,752 --> 01:13:01,621 You know, there was a person who recently quit a job at a very 1416 01:13:01,621 --> 01:13:04,935 fancy startup and joined another company that is, I can't name it 1417 01:13:04,935 --> 01:13:08,004 by name, but it's a company you've heard of and it's a 1418 01:13:08,004 --> 01:13:10,349 company that builds software that is actually very useful. 1419 01:13:10,939 --> 01:13:14,886 And I think it benefit from the sort of Canva effect that people 1420 01:13:14,886 --> 01:13:16,796 just go and use it because it's cheap. 1421 01:13:16,796 --> 01:13:18,576 They're not gonna shop for alternatives. 1422 01:13:18,606 --> 01:13:21,680 It's doing really well. And she was telling me that she 1423 01:13:21,680 --> 01:13:23,456 was actually, she was a bit concerned. 1424 01:13:23,456 --> 01:13:26,613 She was like, oh my God, I'm going from this alleged unicorn 1425 01:13:26,613 --> 01:13:27,866 which is never happening, you know. 1426 01:13:28,586 --> 01:13:31,766 The company has never actually turned it into a unicorn, the 1427 01:13:31,766 --> 01:13:34,766 stock options are worthless. And then she goes into this. 1428 01:13:34,766 --> 01:13:37,904 But it's actually been very exciting for her to work on a 1429 01:13:37,904 --> 01:13:40,076 product that people use, even if it's not sexy. 1430 01:13:40,106 --> 01:13:41,486 'Cause this is not a sexy product. 1431 01:13:41,899 --> 01:13:44,406 So don't forget about the boring stuff. 1432 01:13:44,406 --> 01:13:45,974 So that would be my second thing. 1433 01:13:46,274 --> 01:13:48,698 And then the third one, which I already said, but I think I 1434 01:13:48,698 --> 01:13:52,130 would like to go back to that, is that if you are building 1435 01:13:52,130 --> 01:13:55,902 anything based on AI, you should acknowledge the limitations of 1436 01:13:55,902 --> 01:13:59,300 AI from the very beginning and embed them in the product. 1437 01:13:59,390 --> 01:14:01,550 And that's how you're gonna build a successful product, 1438 01:14:01,550 --> 01:14:03,360 right? If AI hallucinates, maybe do 1439 01:14:03,360 --> 01:14:06,770 something so that, again, you quote a passage from the text, 1440 01:14:06,770 --> 01:14:09,805 you do not interpret it. You don't leave room for 1441 01:14:09,805 --> 01:14:11,605 hallucinations. Um, I've spoken to companies 1442 01:14:11,605 --> 01:14:14,485 that are doing a good job in this sense, right? 1443 01:14:14,485 --> 01:14:17,121 They are using, or they're building an AI-based product 1444 01:14:17,121 --> 01:14:20,815 that helps, that generally helps save time for people without 1445 01:14:20,815 --> 01:14:24,735 pretending that it can do the job in a flawless way and people 1446 01:14:24,735 --> 01:14:27,475 are liking the product, right? And I think that's something you 1447 01:14:27,475 --> 01:14:28,645 need to consider. Yeah. 1448 01:14:28,675 --> 01:14:30,985 So those are my three. Wow. 1449 01:14:31,015 --> 01:14:34,700 So I find it really insightful and very beautifully said as 1450 01:14:34,700 --> 01:14:36,490 well. So the boring stuff, I think I 1451 01:14:36,490 --> 01:14:39,745 like it the most, right? Because sometimes we are so much 1452 01:14:39,745 --> 01:14:42,685 into the hype, right? So we chase all these hypes. 1453 01:14:42,711 --> 01:14:44,751 But definitely there are things that are still working. 1454 01:14:44,751 --> 01:14:47,331 Maybe it's boring, but it's useful and people use, and it 1455 01:14:47,331 --> 01:14:50,278 creates value in the world. So I think thanks for reminding 1456 01:14:50,278 --> 01:14:53,302 us those things. So Emmanuel, if people like this 1457 01:14:53,302 --> 01:14:56,116 conversation, they wanna reach out to you, ask you more 1458 01:14:56,116 --> 01:14:58,126 questions or discuss about, you know, AI stuff. 1459 01:14:58,206 --> 01:15:00,006 Is there a place where they can find you online? 1460 01:15:00,742 --> 01:15:03,566 Best place is LinkedIn. I usually post stuff on 1461 01:15:03,566 --> 01:15:05,411 LinkedIn. I, yeah, that's where I'm, 1462 01:15:05,411 --> 01:15:08,066 that's where I hang out. I know it's a little bit weird. 1463 01:15:08,066 --> 01:15:11,104 Some people will tell me, you know, why are you on LinkedIn 1464 01:15:11,104 --> 01:15:14,258 and not on Twitter? But that's the main place for me 1465 01:15:14,258 --> 01:15:17,014 to communicate with people. Um, and, uh, you can also find 1466 01:15:17,014 --> 01:15:19,892 my books to learn more about the things I have to say. 1467 01:15:19,892 --> 01:15:22,534 And you can send me a message on LinkedIn. 1468 01:15:23,334 --> 01:15:25,233 Yeah. Right. 1469 01:15:25,713 --> 01:15:28,673 So I highly suggest people reading your book, especially if 1470 01:15:28,673 --> 01:15:31,743 you want to demystify the AI hypes that is happening now. 1471 01:15:31,743 --> 01:15:35,122 Like try to understand the power of it, the fundamentals of it, 1472 01:15:35,122 --> 01:15:37,447 and the pitfalls where AI might fail. 1473 01:15:37,807 --> 01:15:40,379 So thank you so much for writing those books Emmanuel, and for 1474 01:15:40,379 --> 01:15:42,487 this conversation. So I hope you enjoy this 1475 01:15:42,487 --> 01:15:43,886 conversation and thank you again. 1476 01:15:43,967 --> 01:15:44,477 Thank you.