1 00:00:18,680 --> 00:00:23,120 You're working on iPhone Pago, which is the fintech arm of I 2 00:00:23,120 --> 00:00:24,160 Food. Exactly. 3 00:00:24,160 --> 00:00:28,320 So a lot of people know I food, especially in Brazil, most of 4 00:00:28,320 --> 00:00:32,200 Brazilians know I food, it's a food delivery app in the 5 00:00:32,200 --> 00:00:36,000 marketplace, the the market leader in Brazil. 6 00:00:37,200 --> 00:00:40,400 But a few people know that we have all these other business. 7 00:00:40,400 --> 00:00:43,040 One of those is our fintech business. 8 00:00:43,320 --> 00:00:47,920 So our fintech business is how can we help and enable our 9 00:00:47,920 --> 00:00:52,440 restaurants to grow and have more support to grow inside our 10 00:00:52,440 --> 00:00:55,680 platform. So it's, it's not a bank that 11 00:00:55,680 --> 00:01:01,000 lives alone in his space, but it's a bank that serves ifood in 12 00:01:01,000 --> 00:01:04,640 the ifood ecosystem. So success for our bank, our 13 00:01:04,640 --> 00:01:09,000 fintech is seeing our customers, our restaurants grow inside the 14 00:01:09,000 --> 00:01:12,080 platform. And I'm assuming you know a lot 15 00:01:12,080 --> 00:01:16,560 about the restaurants, so you can help them in different ways 16 00:01:16,920 --> 00:01:21,600 by giving them loans, but also understanding their credit and 17 00:01:21,600 --> 00:01:24,400 how risky the loans are. We know how the restaurants 18 00:01:24,400 --> 00:01:27,600 operate. So we can we can give loans to 19 00:01:27,600 --> 00:01:32,400 restaurants that probably wouldn't get a loan outside our 20 00:01:32,400 --> 00:01:35,880 fintech, but we can give this loan because we know that he's a 21 00:01:35,880 --> 00:01:38,760 good restaurant, that he's serving the clients, that he's 22 00:01:38,920 --> 00:01:41,000 actually delivering the food and so on. 23 00:01:41,000 --> 00:01:42,320 But. Why wouldn't they be able to get 24 00:01:42,320 --> 00:01:45,400 it outside? Because around 80% of our 25 00:01:45,400 --> 00:01:48,480 restaurants are actually small restaurants in Brazil. 26 00:01:48,680 --> 00:01:52,120 So sometimes you see a single person that takes care of the 27 00:01:52,120 --> 00:01:55,720 restaurant, the packaging, preparing the meal and then 28 00:01:56,240 --> 00:01:58,680 taking care of the finance of the business as well. 29 00:01:58,680 --> 00:02:02,880 So it's a very small business and sometimes they don't have 30 00:02:02,880 --> 00:02:05,400 access to credit in in regular banks. 31 00:02:05,640 --> 00:02:09,240 So what we do is since we know this is a good restaurant that 32 00:02:09,240 --> 00:02:13,960 is really operating inside eye food, we can give a loan to that 33 00:02:13,960 --> 00:02:16,520 person. And you can know what their 34 00:02:16,520 --> 00:02:20,080 order volume is, and you can understand like, yeah, this is 35 00:02:20,080 --> 00:02:23,360 sustainable. Exactly, because having working 36 00:02:23,360 --> 00:02:25,440 capital is very important for a restaurant. 37 00:02:25,440 --> 00:02:29,120 So they cannot grow if they don't have access to some 38 00:02:29,120 --> 00:02:32,880 capital to start growing the business. 39 00:02:32,960 --> 00:02:34,720 So this is very important for them. 40 00:02:35,040 --> 00:02:41,480 And since we know how it operates and how it's actually 41 00:02:41,480 --> 00:02:45,560 working in the platform, we can choose the right amount, the the 42 00:02:45,560 --> 00:02:47,960 right size of loan for this restaurant. 43 00:02:48,120 --> 00:02:50,120 Yeah. And how are you bringing the 44 00:02:50,480 --> 00:02:53,880 agentic experience into that whole life cycle? 45 00:02:54,160 --> 00:03:00,040 Yeah, we have a lot of different agents inside Pago in that life 46 00:03:00,040 --> 00:03:03,240 cycle. Specifically, we have chat bank, 47 00:03:03,240 --> 00:03:07,840 which, which is like the bank manager for the, for those 48 00:03:07,840 --> 00:03:10,640 restaurants. And it's a a different kind of 49 00:03:10,640 --> 00:03:16,280 bank manager because it's not only saying things about like 50 00:03:16,280 --> 00:03:23,360 your, your balance or general information about your account, 51 00:03:23,360 --> 00:03:26,280 but it actually helps the entrepreneur to run the 52 00:03:26,280 --> 00:03:28,840 restaurant. So it says, OK, you need to pay 53 00:03:28,840 --> 00:03:32,720 your employees. The, the, the entrepreneur can 54 00:03:32,720 --> 00:03:34,720 say, I want to play my employees. 55 00:03:34,720 --> 00:03:39,240 I have these five employees and those are the accounts. 56 00:03:39,560 --> 00:03:43,440 And then the, the bank manager can do the, the bank transfers. 57 00:03:44,640 --> 00:03:46,120 It also can and. The bank. 58 00:03:46,360 --> 00:03:50,000 So me as a restaurant owner, I keep all my money in the bank 59 00:03:50,440 --> 00:03:57,440 and I'm using ifood as my bank as opposed to whatever like ING 60 00:03:57,440 --> 00:04:01,400 or N 26 type of thing. Different people choose 61 00:04:01,400 --> 00:04:05,600 different ways of working, but a lot of our restaurants use our 62 00:04:05,600 --> 00:04:09,440 account because of all these different features that we have 63 00:04:09,440 --> 00:04:10,680 for. Them it makes it a lot easier I 64 00:04:10,680 --> 00:04:12,160 can imagine. Yes, exactly. 65 00:04:12,160 --> 00:04:17,440 So this chat bank assistant the the bank manager, it helps the 66 00:04:18,320 --> 00:04:22,480 the the restaurant owner with the day-to-day life of the 67 00:04:22,480 --> 00:04:25,640 restaurant. So making payments, knowing that 68 00:04:25,640 --> 00:04:28,680 he has an outstanding invoice that he needs to pay. 69 00:04:29,000 --> 00:04:33,320 And this also is an opportunity for helping him with the working 70 00:04:33,320 --> 00:04:36,320 capital as well. So for example, we can say we 71 00:04:36,320 --> 00:04:39,800 see that you have an outstanding payment, but you don't have the 72 00:04:39,800 --> 00:04:43,240 money in your account right now. But I, I see that you have 73 00:04:43,240 --> 00:04:46,640 access to credit like you have this three different sources of 74 00:04:46,640 --> 00:04:49,800 credit you can use. But I'm seeing that the cheapest 75 00:04:49,800 --> 00:04:53,320 1 is this one. So you can access the cheapest 76 00:04:53,320 --> 00:04:59,280 possible credit just to solve your problem right now in the in 77 00:04:59,280 --> 00:05:01,160 the quickest and cheapest way possible. 78 00:05:01,320 --> 00:05:03,440 And this is all done through a chat interface. 79 00:05:03,600 --> 00:05:06,280 Yes, exactly. WhatsApp is very popular in 80 00:05:06,280 --> 00:05:11,040 Brazil and the restaurant owners are usually operating in 81 00:05:11,040 --> 00:05:14,120 WhatsApp as well. So he's talking to suppliers, 82 00:05:14,320 --> 00:05:16,400 he's talking to customers in WhatsApp. 83 00:05:16,760 --> 00:05:21,040 So helping him in WhatsApp works really well because it's it's 84 00:05:21,040 --> 00:05:24,640 very comfortable for him. So they, I imagine a world where 85 00:05:24,640 --> 00:05:29,000 they can just forward invoices to chat, Bing, WhatsApp, and 86 00:05:29,000 --> 00:05:31,640 then it's automatically uploaded, Yeah. 87 00:05:31,640 --> 00:05:32,800 Exactly. He actually does that. 88 00:05:32,800 --> 00:05:33,440 Yeah. Yeah. 89 00:05:33,440 --> 00:05:37,080 He can even take a picture of something and say I want to make 90 00:05:37,080 --> 00:05:41,400 a pix for this number. Pix is the Brazilian Trans Money 91 00:05:41,400 --> 00:05:43,960 Transfer Protocol, one of the the protocols. 92 00:05:44,160 --> 00:05:47,400 So I can say I just want to transfer money for this account 93 00:05:47,400 --> 00:05:50,160 and take a picture and then the Chet Bank takes care of the 94 00:05:50,160 --> 00:05:50,640 rest. Yeah. 95 00:05:51,320 --> 00:05:56,000 It's like pay this shit, I don't want to deal with it Exactly and 96 00:05:56,000 --> 00:06:01,640 it will transfer the funds. It's just I send a WhatsApp 97 00:06:01,640 --> 00:06:04,360 message. Yes, and then you make the 98 00:06:04,360 --> 00:06:07,440 transfer. It's, of course, it's a secure 99 00:06:07,440 --> 00:06:11,240 procedure, so the the restaurant owner has to tap his password, 100 00:06:11,240 --> 00:06:14,520 but everything is very well integrated, like the user 101 00:06:14,520 --> 00:06:18,120 experience is very seamless for him. 102 00:06:18,680 --> 00:06:21,640 So he just types the password and then the transfer is made. 103 00:06:21,640 --> 00:06:24,920 Wow, yeah. Behind the scenes, what's it 104 00:06:24,920 --> 00:06:27,320 look like? Behind the scenes we have an 105 00:06:27,320 --> 00:06:32,280 agent there that for every message the user sends it reacts 106 00:06:32,280 --> 00:06:35,000 to it. So it it receives the message 107 00:06:35,280 --> 00:06:39,520 creates an execution plan. So what should I do for the this 108 00:06:39,520 --> 00:06:43,760 message? So for example, I need to start 109 00:06:44,040 --> 00:06:49,400 a money transfer operation. So he calls this function and 110 00:06:49,400 --> 00:06:56,280 then this function becomes UI element inside WhatsApp for the 111 00:06:56,280 --> 00:06:59,320 user to confirm the password. And then the after the 112 00:06:59,320 --> 00:07:02,920 password's confirmed, then the transfer is created in the back 113 00:07:02,920 --> 00:07:06,200 end. So it's the agent works like a 114 00:07:06,200 --> 00:07:10,520 front end, like a web front end. And you have all these tools in 115 00:07:10,520 --> 00:07:13,720 the back end that interface with the different APIs we have 116 00:07:13,720 --> 00:07:19,720 inside the the fintech. So we have APIs for the money 117 00:07:19,720 --> 00:07:24,320 transfer, we have APIs for getting the account balance for 118 00:07:24,320 --> 00:07:28,040 authentication. We need to know if a WhatsApp 119 00:07:28,040 --> 00:07:32,880 number is attached to a fintech account, what account is that 120 00:07:32,880 --> 00:07:35,760 account. So these are all the different 121 00:07:35,760 --> 00:07:41,720 back end functions that we need and and we have other agents and 122 00:07:41,720 --> 00:07:45,240 other business as well. So inside our Fintech we have 123 00:07:45,240 --> 00:07:49,640 beneficious which is a corporate meal voucher that is very common 124 00:07:49,640 --> 00:07:52,840 in Brazil. It's like a credit card but only 125 00:07:52,840 --> 00:07:58,240 for meals. And we have an agent that helps 126 00:07:58,360 --> 00:08:02,960 small companies hire this the service and also operate the 127 00:08:02,960 --> 00:08:07,200 service in a day-to-day basis. And hire the service? 128 00:08:07,200 --> 00:08:11,240 You mean like go and get you a credit card? 129 00:08:12,640 --> 00:08:16,160 Not for the person, but the company's HR. 130 00:08:16,440 --> 00:08:20,040 So I'm from a company HR and I go there and I want to hire this 131 00:08:20,040 --> 00:08:24,280 beneficious for all my employees and have this credit card for 132 00:08:24,280 --> 00:08:26,960 all my employees. And the thing with the agents 133 00:08:26,960 --> 00:08:33,320 here is that we can scale a very custom experience for this 134 00:08:33,799 --> 00:08:35,919 person. So we can say, OK, what's the 135 00:08:35,919 --> 00:08:40,200 size of your company? What are your needs for 136 00:08:40,200 --> 00:08:42,880 beneficious for the the meal voucher? 137 00:08:43,480 --> 00:08:47,800 What are you worried about? And sometimes people ask what's 138 00:08:47,800 --> 00:08:50,840 the difference between this product here and this other 139 00:08:50,840 --> 00:08:54,040 product? And then we can help the user 140 00:08:54,320 --> 00:08:57,040 guide his decision and and make the best decision. 141 00:08:57,040 --> 00:08:59,480 So this is one of the ways we use these agents. 142 00:08:59,640 --> 00:09:04,280 Yeah, talk to me more about this idea of scaling personalization. 143 00:09:04,560 --> 00:09:07,880 Yeah, because I think that's something fascinating that you 144 00:09:07,920 --> 00:09:13,000 all are doing and it is mainly happening right now through 145 00:09:13,000 --> 00:09:14,560 chat, right? Yeah. 146 00:09:15,120 --> 00:09:17,680 But what are your thoughts around it? 147 00:09:18,440 --> 00:09:22,080 Yeah. So for me, when I started using 148 00:09:22,640 --> 00:09:27,440 AI, when I started working with AI, the the interesting thing 149 00:09:27,440 --> 00:09:33,480 for me is that you can bring a very customized experience to a 150 00:09:33,480 --> 00:09:37,440 whole new level. You can like scale that to a 151 00:09:37,440 --> 00:09:41,200 huge population. So for example, when you look at 152 00:09:41,360 --> 00:09:45,560 ifood, not only the fintech part, but but the whole ifood 153 00:09:46,240 --> 00:09:48,960 company, you have a lot of opportunities that you want to 154 00:09:48,960 --> 00:09:53,040 be more tailored to the customers. 155 00:09:53,040 --> 00:09:56,640 So when you're ordering food, I want to offer you what's good 156 00:09:56,640 --> 00:10:01,120 for you. When I'm trying to help you find 157 00:10:01,200 --> 00:10:06,240 and hire beneficials, I want to clarify for you and make easier 158 00:10:06,240 --> 00:10:09,040 for you to understand how beneficious can help you in your 159 00:10:09,040 --> 00:10:11,840 company, not someone else's company. 160 00:10:12,600 --> 00:10:18,160 And with agents, we can bring that to everyone so you can talk 161 00:10:18,160 --> 00:10:21,280 to the person in their own context. 162 00:10:21,440 --> 00:10:25,240 So it's way easier to do that. And today this is, this mostly 163 00:10:25,240 --> 00:10:30,040 happens by chat because it's a very open-ended interface. 164 00:10:30,040 --> 00:10:34,920 It's, it's kind of easy to put everywhere, but what I'm seeing 165 00:10:34,920 --> 00:10:40,280 right now is more and more we're starting to creating like custom 166 00:10:40,360 --> 00:10:43,840 UIS that are more tailored to the specific cases. 167 00:10:43,840 --> 00:10:49,120 So for example, we, we have a case in ILO, which is our food 168 00:10:49,120 --> 00:10:55,600 recommendation agent that having the the right UI for the user 169 00:10:55,600 --> 00:10:59,080 not only saves time because he can just tap the dish that he 170 00:10:59,080 --> 00:11:02,680 wants, but it also saves a lot of processing in the background 171 00:11:02,880 --> 00:11:05,720 because reduce like LLM calls. Yeah. 172 00:11:05,960 --> 00:11:09,680 So it's not only about having the right experience, which is 173 00:11:09,680 --> 00:11:15,320 easier and and and nicer to use, but it's also about efficiency 174 00:11:15,360 --> 00:11:17,400 as well. Dude that is so true. 175 00:11:17,400 --> 00:11:23,360 The tightrope that you have to walk to make sure that you're 176 00:11:23,360 --> 00:11:27,800 not just sending everything to the LLM, It's that fine line and 177 00:11:27,800 --> 00:11:34,080 you get to be almost an artist about where you throw in the LLM 178 00:11:34,160 --> 00:11:38,400 and where you don't. Yes, as the models improves, 179 00:11:38,880 --> 00:11:43,880 it's getting easier because it's not that sensitive to your, 180 00:11:44,240 --> 00:11:47,440 your, the, the way you write your prompt and everything. 181 00:11:47,680 --> 00:11:51,280 But there's still some magic to it, like maybe it's one word 182 00:11:51,280 --> 00:11:54,480 that you put that changes how the agent works. 183 00:11:55,640 --> 00:12:00,880 So how you evaluate that and and how you choose what you put 184 00:12:00,880 --> 00:12:04,120 inside the context is it's very important. 185 00:12:04,320 --> 00:12:06,880 Yeah. And just thinking about where 186 00:12:06,880 --> 00:12:13,240 the human plays the role, where do you actually have the human 187 00:12:13,240 --> 00:12:18,600 be part of the process versus not, is also fascinating in my 188 00:12:18,600 --> 00:12:21,520 mind. Because I imagine you're 189 00:12:21,520 --> 00:12:28,880 thinking a lot about how to get the merchant the least friction 190 00:12:28,880 --> 00:12:33,840 way possible to do something. So for you, anywhere that the 191 00:12:33,840 --> 00:12:38,960 human has to do something is it's like it better be a must 192 00:12:39,200 --> 00:12:43,320 that they have to actually do it, because otherwise you're not 193 00:12:43,320 --> 00:12:47,760 saving them as much time. But but at the same time, you 194 00:12:47,760 --> 00:12:52,120 have to build trust as well. So when you use an agent and it 195 00:12:52,120 --> 00:12:55,600 starts doing everything for you, maybe you can get uncomfortable 196 00:12:55,600 --> 00:12:58,600 like I don't want you to do that or just wait. 197 00:12:58,720 --> 00:13:02,640 You need my approval to do that. So it's it's a fine line you 198 00:13:02,640 --> 00:13:07,120 have to walk between giving the automation and making the person 199 00:13:07,360 --> 00:13:11,520 comfortable with the decisions that are being taken. 200 00:13:11,600 --> 00:13:14,200 Well, especially with money. Yeah, exactly. 201 00:13:14,200 --> 00:13:16,760 So it's it's a very sensitive topic, Yeah. 202 00:13:16,840 --> 00:13:21,400 Yeah, you're, I, I mean, if the wrong pizza shows up at my door, 203 00:13:21,680 --> 00:13:25,720 that's one thing, But if you send my employees an extra 2 204 00:13:25,720 --> 00:13:29,320 grand, I'm gonna. That's a totally different 205 00:13:29,320 --> 00:13:31,840 problem to have. And if it's the agent's mistake 206 00:13:31,840 --> 00:13:35,000 on both of those parts, like 1 is a little excusable. 207 00:13:35,520 --> 00:13:39,320 The other is like, hey, I want that 2 grand back, but I'm not 208 00:13:39,320 --> 00:13:45,040 going to not give it back to my like it's just a messy situation 209 00:13:45,040 --> 00:13:46,520 that you're in. Yeah, exactly. 210 00:13:47,360 --> 00:13:51,360 The the harness we build for different types of products and 211 00:13:51,360 --> 00:13:53,240 applications is completely different. 212 00:13:53,800 --> 00:13:59,680 Like I, I, I like to put agents and AI in a spectrum, like in in 213 00:13:59,680 --> 00:14:04,360 one end, you have a lot of control and in on the other end, 214 00:14:04,360 --> 00:14:07,600 you have a lot of magic. So you have to choose where 215 00:14:07,600 --> 00:14:10,840 you're going to put your product in this spectrum because 216 00:14:10,840 --> 00:14:14,880 sometimes it will feel a bit more stiff, but you want that 217 00:14:14,880 --> 00:14:18,640 because you need more control. And sometimes the the risks are 218 00:14:18,640 --> 00:14:20,680 low. So you want to see more magic. 219 00:14:20,680 --> 00:14:23,800 You want things to happen and and people to feel that wow 220 00:14:23,800 --> 00:14:27,560 moment when using your product. Are there moments where you've 221 00:14:27,560 --> 00:14:31,600 just let it go completely to the magic side of that spectrum? 222 00:14:31,640 --> 00:14:33,840 Yeah. I don't think that in fintech 223 00:14:33,840 --> 00:14:39,400 business we have that, but in I food in ILO, for example, for 224 00:14:40,240 --> 00:14:44,080 recommending food, yes, it we want it to be a magical 225 00:14:44,080 --> 00:14:47,280 experience. So instead of saying I just want 226 00:14:47,280 --> 00:14:51,280 something that I like or I, I just want the pizza here, we 227 00:14:51,280 --> 00:14:56,160 want people to ask what should I get for a romantic dinner with 228 00:14:56,160 --> 00:15:01,760 my wife? So this is a different level of 229 00:15:02,200 --> 00:15:04,200 openness that we want from our users. 230 00:15:04,720 --> 00:15:08,520 You talked a little bit about how the chat interface is 231 00:15:08,520 --> 00:15:12,320 changing and how we're starting to create these new components. 232 00:15:12,720 --> 00:15:17,360 I'm also fascinated by that. I've seen there's a, there's a 233 00:15:17,360 --> 00:15:22,680 project that I saw on Figma that was, I can't, I think it's 234 00:15:22,680 --> 00:15:27,440 called Language GUI and it's a bunch of Figma components that 235 00:15:27,440 --> 00:15:33,280 you can use inside the chat to help render different things 236 00:15:33,280 --> 00:15:35,160 that you wouldn't necessarily get in a chat. 237 00:15:35,160 --> 00:15:39,680 Maybe that's an embedded tweet or it's a voice note or it's a 238 00:15:39,680 --> 00:15:44,640 snippet of a podcast, so that instead of always getting back 239 00:15:45,520 --> 00:15:51,280 text, I get different, more enriched data points. 240 00:15:51,280 --> 00:15:56,240 And now we're seeing that a ton with the MCPUI, which is that 241 00:15:56,240 --> 00:15:58,320 taking to the extreme. And I'm excited to see what 242 00:15:58,320 --> 00:16:00,840 people build with it. I know that some people in the 243 00:16:00,840 --> 00:16:04,720 MO OPS community, Slack, we're talking about how hard it was to 244 00:16:04,720 --> 00:16:08,400 work with MCPUI right now, just because it's so nascent and you 245 00:16:08,400 --> 00:16:13,360 kind of expect that. But are you thinking through 246 00:16:13,360 --> 00:16:17,080 different components that you're now rendering? 247 00:16:17,080 --> 00:16:22,120 You talked about how you tap for authorization, so that's one. 248 00:16:22,800 --> 00:16:26,120 What else is there that you're like trying to bring into the 249 00:16:26,120 --> 00:16:29,800 chat that you don't have to have the user leave for any reason? 250 00:16:29,960 --> 00:16:35,160 Those are usually very specific and tailor made for each case. 251 00:16:35,400 --> 00:16:39,520 Of course you have like a generic, a genetic UI toolkit 252 00:16:39,520 --> 00:16:45,240 that can be used. But I think the the real 253 00:16:45,480 --> 00:16:49,240 experience it, it becomes really good when you tailor the 254 00:16:49,240 --> 00:16:52,920 components exactly for the use case that we have and what the 255 00:16:52,920 --> 00:16:56,800 user needs. So it's, it's not about the only 256 00:16:56,800 --> 00:17:01,880 about the agentic part, the AI, but it's you, it's the how the 257 00:17:02,040 --> 00:17:06,440 the customer is using your product in a day-to-day basis in 258 00:17:06,440 --> 00:17:10,400 the experience that he has. So those special components are 259 00:17:10,400 --> 00:17:13,160 very important. I think we have this for 260 00:17:13,960 --> 00:17:18,040 passwords, for confirming information with the user to 261 00:17:18,040 --> 00:17:21,400 make sure that he's aware of what the agent is doing and 262 00:17:21,400 --> 00:17:23,640 what's happening. So this is something we are 263 00:17:23,640 --> 00:17:29,080 experimenting about. But when I think of agentic UI 264 00:17:29,080 --> 00:17:33,720 components and so on, I also see that spectrum from control to 265 00:17:33,720 --> 00:17:36,240 magic. Like on one side you have all 266 00:17:36,240 --> 00:17:39,960 these agentic UI toolkits popping up right now with 267 00:17:40,000 --> 00:17:45,320 different CSSHTML or React components already packed that 268 00:17:45,320 --> 00:17:48,720 the agent can choose. And on the other side of the 269 00:17:48,720 --> 00:17:54,600 spectrum you have like free HTML with JavaScript and CSS with 270 00:17:54,760 --> 00:17:58,600 which also is working really well nowadays. 271 00:17:58,600 --> 00:18:02,480 So for for some internal tools we are actually doing that a 272 00:18:02,480 --> 00:18:06,400 lot. So generate on the fly the HTML 273 00:18:06,400 --> 00:18:10,680 with this visualization for this specific case. 274 00:18:10,760 --> 00:18:13,640 Yeah, that's kind of like the playgrounds that I was showing 275 00:18:13,640 --> 00:18:14,080 you. Yeah. 276 00:18:14,080 --> 00:18:18,200 Exactly, Yeah, like the playgrounds you were showing, 277 00:18:18,400 --> 00:18:25,920 for example, we had one one problem in a in a, in a data set 278 00:18:25,920 --> 00:18:29,320 that we had and we wanted to investigate in the book what was 279 00:18:29,320 --> 00:18:31,840 happening. And we created a simple tool 280 00:18:31,840 --> 00:18:33,840 like called code created this tool. 281 00:18:33,840 --> 00:18:36,280 I want to investigate this, this and that. 282 00:18:36,280 --> 00:18:40,680 I want to see the lineage of the data set and why this happened. 283 00:18:40,680 --> 00:18:44,080 And it just generates the HTML with the visualization, the 284 00:18:44,080 --> 00:18:46,280 graph with the data sets being built. 285 00:18:46,640 --> 00:18:49,520 And it was very easy to see why the problem happened. 286 00:18:49,560 --> 00:18:54,240 Yeah, I, I've, that's one of my primary use cases for this 287 00:18:54,240 --> 00:18:56,960 playground skill is debugging, Yeah. 288 00:18:57,400 --> 00:19:00,280 And getting visual representations and then being 289 00:19:00,280 --> 00:19:04,000 able to hone in on what the problem is and where it's 290 00:19:04,000 --> 00:19:06,680 happening. And then you can just copy over 291 00:19:06,680 --> 00:19:09,280 the prompt and be like alright fix that for me please. 292 00:19:09,600 --> 00:19:12,080 Yeah, exactly. It's a really cool application. 293 00:19:12,240 --> 00:19:16,160 When you're talking about these different components that you're 294 00:19:16,160 --> 00:19:19,200 serving up to customers, do you feel like that is also 295 00:19:19,200 --> 00:19:23,600 personalization at scales? So now I'm going to get on the 296 00:19:23,600 --> 00:19:27,640 app and it knows the top four things that I normally do when I 297 00:19:27,640 --> 00:19:30,000 talk to it on WhatsApp or whatever. 298 00:19:30,000 --> 00:19:32,280 So it instantly says, hey, welcome back. 299 00:19:32,280 --> 00:19:36,080 Here's do you want to do any of these, like quick buttons, you 300 00:19:36,080 --> 00:19:37,680 know, so I don't even have to think. 301 00:19:37,760 --> 00:19:41,000 I'm talking more about context in general. 302 00:19:41,040 --> 00:19:44,480 That affects everything. So for example, I know that you 303 00:19:44,480 --> 00:19:48,080 have two employees. I know that you're not opening 304 00:19:48,080 --> 00:19:51,320 your restaurant tomorrow. Those are the things in context 305 00:19:51,320 --> 00:19:55,800 that start summing up to give a nice experience for the user. 306 00:19:56,000 --> 00:19:59,920 So today I'm not going to say for I'm not going to send you a 307 00:19:59,920 --> 00:20:03,560 message for you to pay your employees because I know you're 308 00:20:03,560 --> 00:20:05,400 not opening today or something like that. 309 00:20:05,400 --> 00:20:09,440 So this context enriches every experience. 310 00:20:10,240 --> 00:20:14,080 And and one being the the buttons I'm going to show in 311 00:20:14,080 --> 00:20:17,760 what order and what's the user going to use first? 312 00:20:17,760 --> 00:20:20,280 Like what's the most frequently used? 313 00:20:20,840 --> 00:20:24,400 Yeah. Right now, behind the scenes to 314 00:20:24,640 --> 00:20:28,600 create that context. I know that isn't the easiest 315 00:20:28,600 --> 00:20:31,480 problem. How are you going about making 316 00:20:31,480 --> 00:20:35,280 sure that you have all of the right context at the right time? 317 00:20:36,000 --> 00:20:37,640 Yeah. I think it's something that is 318 00:20:37,640 --> 00:20:42,960 more of a emergent behavior because you have different 319 00:20:42,960 --> 00:20:45,920 systems. Some are pretty simple but bring 320 00:20:45,920 --> 00:20:48,920 this feeling and some are pretty complex. 321 00:20:49,360 --> 00:20:52,600 In in one end of the spectrum, the complex part, we have LCM, 322 00:20:52,600 --> 00:20:58,120 which is our own custom LLM tailored for e-commerce data 323 00:20:58,120 --> 00:21:02,480 that has the profiles that can build profiles for the user and 324 00:21:02,480 --> 00:21:08,720 say Thiago really likes Japanese food and during the weekends he 325 00:21:08,720 --> 00:21:10,960 likes to eat pizza as well and so on. 326 00:21:10,960 --> 00:21:14,240 So this is LCM that brings a whole new level of 327 00:21:14,240 --> 00:21:18,640 personalization for our customers in I food. 328 00:21:18,840 --> 00:21:22,560 But on the other end of the spectrum, maybe just counting 329 00:21:22,560 --> 00:21:27,800 how frequently you you use every feature and use that to sort the 330 00:21:27,960 --> 00:21:30,960 the menu is enough. So I think this is more of a 331 00:21:31,360 --> 00:21:35,400 emergent behavior that when we start assembling all the pieces 332 00:21:35,400 --> 00:21:39,480 and putting everything together, it seems like it's pretty 333 00:21:39,480 --> 00:21:44,000 complex, but some components are complex, some are very simple 334 00:21:44,200 --> 00:21:47,800 and they when they are all aligned, they bring this this 335 00:21:47,800 --> 00:21:53,200 feeling. Is there stuff with like your 336 00:21:53,200 --> 00:21:56,720 context engineering that is the low hanging fruit? 337 00:21:57,040 --> 00:22:01,720 Usually it's the the basic profile that you have on the 338 00:22:01,720 --> 00:22:04,240 restaurant or the person and so on. 339 00:22:04,560 --> 00:22:08,280 So for example, I'll give an example with the the customer 340 00:22:08,280 --> 00:22:11,440 facing part because I think it's easier to visualize. 341 00:22:11,440 --> 00:22:16,560 But if I have my last 10 orders inside my context, it's pretty 342 00:22:16,560 --> 00:22:21,800 easy to see what I like eating. So for the restaurants, this can 343 00:22:21,800 --> 00:22:24,720 be the, the, the way it operates. 344 00:22:24,960 --> 00:22:31,800 So where it is located, what's the opening hours and what's the 345 00:22:31,800 --> 00:22:37,160 cuisine, all these things can help creating this experience 346 00:22:37,160 --> 00:22:39,120 for the restaurant. So for example, we have a 347 00:22:39,280 --> 00:22:43,080 product which is I food shop that sells supplies for 348 00:22:43,080 --> 00:22:46,720 restaurants. And knowing the cuisine is and 349 00:22:46,720 --> 00:22:51,040 the dishes that the restaurant serves is the information we use 350 00:22:51,040 --> 00:22:54,480 to recommend what products he can buy. 351 00:22:54,480 --> 00:22:58,200 So for example, if it's a pizza restaurant, I'm going to sell, 352 00:22:58,200 --> 00:23:01,680 for example, cheese because I know it's something that he uses 353 00:23:01,680 --> 00:23:02,920 a lot. Yeah, you're not going to 354 00:23:02,920 --> 00:23:04,840 recommend chopsticks. Yes, exactly. 355 00:23:05,240 --> 00:23:10,600 Yeah, it feels like there is a law of diminishing returns with 356 00:23:11,000 --> 00:23:15,040 the context that you give it versus what actually is the most 357 00:23:15,040 --> 00:23:19,080 valuable in that output and what your your end user is 358 00:23:19,080 --> 00:23:22,880 experiencing. Depends a lot on the task you're 359 00:23:22,880 --> 00:23:28,520 actually trying to solve. So when when I'm building 360 00:23:28,520 --> 00:23:32,840 agents, I see that if the technology was perfect and the 361 00:23:32,840 --> 00:23:38,120 models were, were perfect, we would we would have only one 362 00:23:38,120 --> 00:23:43,600 single agent agentic prompt that says everything the agent has to 363 00:23:43,600 --> 00:23:47,400 do with all the guides and all the information and all the 364 00:23:47,400 --> 00:23:49,280 tools. And it would have access to like 365 00:23:49,280 --> 00:23:51,560 100 tools and everything would be perfect. 366 00:23:52,000 --> 00:23:53,640 But the reality is not like that. 367 00:23:53,640 --> 00:23:56,800 As you increase the prompt size, as you increase the number of 368 00:23:56,800 --> 00:24:00,120 different instructions the agents has to follow, as you add 369 00:24:00,120 --> 00:24:05,840 more tools, it gets harder and harder for the agent to to have 370 00:24:05,840 --> 00:24:08,440 a high accuracy to do the right thing at the right time. 371 00:24:08,880 --> 00:24:13,720 So what we see is that as the complexity grows, you start 372 00:24:13,720 --> 00:24:20,960 breaking this agent in sub graphs of agents that that 373 00:24:20,960 --> 00:24:23,920 solves a specific problem. So for example, this agent 374 00:24:23,920 --> 00:24:27,560 starts really complex. I see that when I'm talking 375 00:24:27,560 --> 00:24:31,920 about payments for the employees, I, I should have a 376 00:24:31,920 --> 00:24:34,480 special care when dealing with this part. 377 00:24:34,480 --> 00:24:38,280 So I've created a different prompt of a different context 378 00:24:38,280 --> 00:24:41,240 with different tools for the agent to handle that specific 379 00:24:41,240 --> 00:24:44,120 case. So when we look at the context, 380 00:24:44,120 --> 00:24:46,600 for me this is very important, what we're trying to solve. 381 00:24:46,960 --> 00:24:51,240 And this context is going to change given the task we're 382 00:24:51,240 --> 00:24:54,880 trying to solve. Do you feel like there is ever a 383 00:24:54,880 --> 00:24:59,680 point where you're just going to have sub agent bloat because 384 00:24:59,680 --> 00:25:04,760 you've created so many sub agents now that the at some 385 00:25:04,760 --> 00:25:08,240 point there's got to be that bottleneck, right? 386 00:25:08,280 --> 00:25:12,400 And so if Or is it just layers of abstraction? 387 00:25:13,280 --> 00:25:16,160 I see more as layers of abstraction. 388 00:25:16,160 --> 00:25:19,720 It's like breaking a complex problem in smaller problems. 389 00:25:20,160 --> 00:25:23,360 So it's usually easier to handle that way. 390 00:25:23,520 --> 00:25:27,240 So I haven't reached a point of a sub agent bloat or something 391 00:25:27,240 --> 00:25:29,720 like that. And I don't even like to think a 392 00:25:29,720 --> 00:25:35,080 lot as sub agents because when we're building an agent, I see 393 00:25:35,080 --> 00:25:39,080 more like a graph like I have these nodes and this node 394 00:25:39,280 --> 00:25:43,920 execute a specific operation and it can go to other nodes and so 395 00:25:43,920 --> 00:25:47,280 on. So a sub agent can be a sub 396 00:25:47,280 --> 00:25:50,600 graph can be a set of different nodes that solve a task. 397 00:25:51,960 --> 00:25:57,520 And it's a very seamless thing is it's not something like you, 398 00:25:57,520 --> 00:26:00,520 you say, oh, I'm going to have those 3 sub agents. 399 00:26:00,520 --> 00:26:04,320 It's more something like grows into 3 sub agents. 400 00:26:04,640 --> 00:26:10,400 So when I'm building a single product, I think more of an 401 00:26:10,400 --> 00:26:14,520 agent that starts to be assembled by different pieces 402 00:26:14,520 --> 00:26:17,880 and start to being created by different pieces. 403 00:26:18,040 --> 00:26:22,080 When I think of sub agents, I I think in something more similar 404 00:26:22,080 --> 00:26:25,240 to like micro services. Like I have two different teams. 405 00:26:25,440 --> 00:26:28,680 One team builds one specific agent to solve a task. 406 00:26:28,680 --> 00:26:32,320 I have another team building an agent that handles another part 407 00:26:32,320 --> 00:26:35,960 of the business and sometimes one will need to call the other 408 00:26:35,960 --> 00:26:38,400 and then I have this extra change of information. 409 00:26:39,320 --> 00:26:43,880 Tell me more about how it grows. If we start with this single 410 00:26:43,880 --> 00:26:47,240 node, that solves everything. This is the way I think all 411 00:26:47,240 --> 00:26:49,840 agents should start. You always start with the 412 00:26:49,960 --> 00:26:53,800 simplest approach possible, which is a single node. 413 00:26:54,680 --> 00:26:58,680 We have access to tools, and then it can choose what to do. 414 00:26:59,120 --> 00:27:03,960 And you have this set of evaluations that for every case 415 00:27:03,960 --> 00:27:06,480 that you have, we evaluate what's the accuracy of your 416 00:27:06,480 --> 00:27:10,280 agent. As you progress and as you get 417 00:27:10,280 --> 00:27:14,520 more data and you start expanding that evaluation set 418 00:27:14,520 --> 00:27:20,240 and you start having real users with your agent, then things 419 00:27:20,240 --> 00:27:23,800 start not working so well. And you start to detect which 420 00:27:23,800 --> 00:27:25,160 points are not working really well. 421 00:27:25,160 --> 00:27:30,600 So when the user tries to do action A, it's usually the, the, 422 00:27:30,720 --> 00:27:33,920 the agent usually messes up what he's trying to do. 423 00:27:34,480 --> 00:27:36,400 So here's something that we should break. 424 00:27:36,680 --> 00:27:39,880 So you go to your evals, you're going to see that in your eval, 425 00:27:40,200 --> 00:27:43,040 this specific case is not performing really well. 426 00:27:43,280 --> 00:27:46,240 And then you break that node in two nodes. 427 00:27:46,240 --> 00:27:49,280 And now I have one node that takes care of only this specific 428 00:27:49,280 --> 00:27:51,240 case. And then you go back to your 429 00:27:51,240 --> 00:27:53,840 evals and see how it's working. Now everything's fine. 430 00:27:54,320 --> 00:27:57,240 We, we keep working with this agent. 431 00:27:58,360 --> 00:28:01,840 As you progress, as you get more data, people use it more than 432 00:28:01,840 --> 00:28:04,520 you're going to identify another use case that's not working 433 00:28:04,520 --> 00:28:06,080 well. And then you do the same thing. 434 00:28:06,080 --> 00:28:10,160 Now I have to break this specific node in two nodes and 435 00:28:10,160 --> 00:28:12,280 so on. So it's more of a iterative 436 00:28:12,560 --> 00:28:15,200 process. It's not something that you, I, 437 00:28:15,960 --> 00:28:18,200 well, I believe it's not something you can plan in 438 00:28:18,200 --> 00:28:20,480 advance. So I'm going to have this agent, 439 00:28:20,480 --> 00:28:24,640 I'm going to have all this 10 nodes and it's going to be the 440 00:28:24,640 --> 00:28:27,880 best architecture possible. I think the architecture is 441 00:28:27,880 --> 00:28:31,600 being discovered as you gain knowledge of the problem. 442 00:28:31,920 --> 00:28:34,880 Yeah, you don't try and boil the ocean with that first agent. 443 00:28:34,880 --> 00:28:37,240 Exactly. You see what it's doing right, 444 00:28:37,240 --> 00:28:40,560 what it's doing wrong, and then you try and create those edge 445 00:28:40,560 --> 00:28:43,640 cases and continue to have that feedback loop. 446 00:28:43,760 --> 00:28:48,240 I I like that vision. Now I understand why you say it 447 00:28:48,240 --> 00:28:52,440 grows because it's that iterative loop and you water 448 00:28:52,440 --> 00:28:57,560 your little plant by giving it more evals and looking at it 449 00:28:57,560 --> 00:28:59,320 again, seeing where we can break it up. 450 00:28:59,640 --> 00:29:04,560 OK, now there's a new branch. Now it's sprouted and and this 451 00:29:04,560 --> 00:29:08,520 side of the tree is flowering. Cool, let's go there. 452 00:29:08,520 --> 00:29:11,640 Let's water that a little bit more and give it more eval sets. 453 00:29:11,760 --> 00:29:15,840 Yeah, in in my opinion, complexity should never be 454 00:29:15,840 --> 00:29:19,400 planned. It it should be a result of of 455 00:29:20,960 --> 00:29:22,760 time. It's not something that you 456 00:29:22,760 --> 00:29:24,960 prepare. You're going to start complex. 457 00:29:24,960 --> 00:29:29,000 You have to reach that point after a lot of time of work. 458 00:29:29,480 --> 00:29:34,840 And and for me, what makes a very effective team at building 459 00:29:34,840 --> 00:29:39,120 agents is actually running this feedback loop really quick. 460 00:29:39,560 --> 00:29:44,840 So if you can get new instance for your evals, run your evals, 461 00:29:44,840 --> 00:29:50,280 detect what's what's wrong, grow in your agent and doing that 462 00:29:50,280 --> 00:29:54,040 loop really quick, those are the teams that are winners and those 463 00:29:54,040 --> 00:29:55,800 are the agents that are winners as well. 464 00:29:56,360 --> 00:30:02,280 I've never met an engineer that told me I wish this system was 465 00:30:02,280 --> 00:30:06,840 more complex. If anything you try and go the 466 00:30:06,840 --> 00:30:09,720 other way and like you said, you want to try and keep it as 467 00:30:09,720 --> 00:30:12,880 simple as possible. But that feedback loop, you 468 00:30:12,880 --> 00:30:17,920 allow it to enrich the experience and complexity comes 469 00:30:17,920 --> 00:30:20,360 from that. But that's only when you're 470 00:30:20,760 --> 00:30:22,800 faced with the reality of the world. 471 00:30:22,920 --> 00:30:25,240 Exactly. It guides you when to add 472 00:30:25,240 --> 00:30:28,720 complexity and where. So I know you had hot takes 473 00:30:28,720 --> 00:30:33,040 yesterday on Claudebot, which is Moldbot, which is open Claude 474 00:30:33,040 --> 00:30:36,240 now. I really like open claw. 475 00:30:36,600 --> 00:30:41,320 It's something that it's, it's a game changer. 476 00:30:41,720 --> 00:30:46,320 It, it doesn't bring a lot of new techniques or anything like 477 00:30:46,320 --> 00:30:50,680 that, but it's a very good interface. 478 00:30:50,680 --> 00:30:55,440 Like you, you have it 24/7 by your side, answering your 479 00:30:55,440 --> 00:30:59,960 questions using the tools. And the, the, the thing that is 480 00:30:59,960 --> 00:31:03,880 different from open Claw is that it actually develops itself. 481 00:31:03,880 --> 00:31:08,120 So for me, this is the the biggest thing for open Claw 482 00:31:08,240 --> 00:31:12,720 because I can say I wanted to create a new SKU to search the 483 00:31:12,720 --> 00:31:17,480 Internet for me and do this. So for I'll give you an example 484 00:31:17,640 --> 00:31:23,120 for my open claw, I asked it to create a new SKU that researches 485 00:31:23,120 --> 00:31:27,440 the Internet and Twitter and see what are the trending topics for 486 00:31:27,440 --> 00:31:31,560 the day. And then it reflects in the art 487 00:31:33,920 --> 00:31:36,840 image to create and it has its consistent style. 488 00:31:36,840 --> 00:31:39,800 So it has like a book of styles that it can use. 489 00:31:40,160 --> 00:31:43,840 And then he chooses for this reflection he's creating for 490 00:31:43,840 --> 00:31:48,240 this specific day, what is the style he's going to use. 491 00:31:48,240 --> 00:31:50,960 And then he generates the description for an image and 492 00:31:50,960 --> 00:31:54,600 then generates an image with this reflection that represents 493 00:31:54,600 --> 00:31:57,120 this reflection. So this is something I created 494 00:31:57,120 --> 00:31:59,480 there. And, and this is not trivial to 495 00:31:59,480 --> 00:32:05,520 do because he had to create a specific script to use the X API 496 00:32:05,520 --> 00:32:08,600 to fetch training data. He had to get data from the 497 00:32:08,600 --> 00:32:13,360 Internet as well, then summarize this, and then he had to create 498 00:32:13,360 --> 00:32:18,440 a script to call the image generation API and so on. 499 00:32:18,640 --> 00:32:22,160 And it's all nicely packaged and I only had to talk to him to 500 00:32:22,160 --> 00:32:25,480 create that. So this self development process 501 00:32:25,640 --> 00:32:29,960 is what makes open class so interesting in my opinion. 502 00:32:30,240 --> 00:32:36,160 But at the same time, all this complexity and all this freedom 503 00:32:36,160 --> 00:32:40,000 for it to create also brings a lot of security risks as well. 504 00:32:40,160 --> 00:32:46,080 So Open Claw expands a lot on the attack surface that we have 505 00:32:47,720 --> 00:32:52,600 not only by installing software and downloading software and and 506 00:32:52,600 --> 00:32:56,200 doing getting skills from somewhere on the Internet and so 507 00:32:56,200 --> 00:33:01,480 on, which is 1 attack surface. But also when you look at prompt 508 00:33:01,480 --> 00:33:05,240 injection and other problems that can arise from processing 509 00:33:05,240 --> 00:33:07,280 the the textual information as well. 510 00:33:07,640 --> 00:33:13,120 And for the prompt injection, it's not enough to control the 511 00:33:13,120 --> 00:33:15,360 entry point. Like for example, if you don't 512 00:33:15,360 --> 00:33:17,520 talk to my open cloud, then I'm safe. 513 00:33:17,680 --> 00:33:21,360 This is not true because maybe you can get a calendar invite 514 00:33:21,360 --> 00:33:23,960 that has a prompt injection, or maybe someone sends you a 515 00:33:23,960 --> 00:33:26,920 message in WhatsApp with a prompt injection, or you receive 516 00:33:26,920 --> 00:33:28,360 an e-mail with a prompt injection. 517 00:33:28,360 --> 00:33:31,560 Yeah, because it's monitoring all of it, Yes, so. 518 00:33:32,080 --> 00:33:38,160 The the amount of excess you give, it's also the amount of 519 00:33:38,160 --> 00:33:41,640 risk you're taking. You had mentioned too before the 520 00:33:41,760 --> 00:33:44,840 idea of how much your job has changed. 521 00:33:45,480 --> 00:33:49,000 As you were talking about what you've created with Open Cloud, 522 00:33:49,200 --> 00:33:52,960 I was like, oh, so basically what you were doing in 2009, you 523 00:33:53,000 --> 00:33:58,080 basically can put together in 20 minutes now with Open Cloud and 524 00:33:58,080 --> 00:34:01,200 probably more. Can you talk me through now like 525 00:34:02,120 --> 00:34:07,000 how you think about working through the all those changes 526 00:34:07,000 --> 00:34:10,840 that you've had over the years? Yeah, I think that the the cost 527 00:34:10,840 --> 00:34:16,600 of software of writing code, it's already a fraction of what 528 00:34:16,600 --> 00:34:20,719 it was a few years ago. We're still in the adaption 529 00:34:20,719 --> 00:34:24,239 period in in which people are actually understand how to use 530 00:34:24,239 --> 00:34:28,600 and how to make it work. But one example for my personal 531 00:34:28,600 --> 00:34:33,080 experience as well, it's been easier sometimes to write a tool 532 00:34:33,080 --> 00:34:36,800 to solve a problem than to find a suitable candidate in in 533 00:34:36,800 --> 00:34:39,840 Google. So for example, I have one 534 00:34:39,840 --> 00:34:43,760 website that I put all the tools, small tools that I code. 535 00:34:44,760 --> 00:34:49,840 One example, I sometimes will create a presentation deck, some 536 00:34:49,840 --> 00:34:53,280 slides and I want to put a video converted as a GIF. 537 00:34:53,280 --> 00:34:56,400 So I want to get a video convert to GIF and put in my 538 00:34:56,400 --> 00:35:00,600 presentation and I always used to search for a website and I 539 00:35:00,640 --> 00:35:04,120 always was thinking if this website is storing my video 540 00:35:04,120 --> 00:35:06,440 there or not and everything which it is. 541 00:35:06,600 --> 00:35:11,800 Yeah. And what I did was I opened 1A 542 00:35:11,800 --> 00:35:15,040 genetic programming tool and I prompted to create. 543 00:35:16,240 --> 00:35:20,480 I want this small tool only using HTML and JavaScript. 544 00:35:20,480 --> 00:35:25,520 So it runs on the browser and you're going to use FFM peg in 545 00:35:25,520 --> 00:35:29,400 web assembly to convert this video to GIF and then it created 546 00:35:29,400 --> 00:35:32,040 like one shot the the application. 547 00:35:32,240 --> 00:35:36,560 So I have this ripple that I put all those tools and I have it 548 00:35:36,560 --> 00:35:41,480 online and for the specific case, it's easier to do that 549 00:35:41,480 --> 00:35:44,000 than to find a suitable candidate. 550 00:35:44,000 --> 00:35:48,520 So right now this is getting very common and as the models 551 00:35:48,920 --> 00:35:54,680 progress and get better, it's going to be more and more common 552 00:35:54,680 --> 00:35:56,880 to see that happen. Actually, you are bringing up 553 00:35:56,880 --> 00:35:59,520 such a good point. Yesterday I saw a tweet from 554 00:35:59,880 --> 00:36:04,360 Andre Caparti. Then he said that he trained a G 555 00:36:04,360 --> 00:36:10,280 PT2 model. It was capable the same stuff 556 00:36:10,520 --> 00:36:14,360 for under 100 bucks. Yeah, that is so crazy because 557 00:36:14,360 --> 00:36:18,080 you think about how much it cost to train back in whatever 2019 558 00:36:18,080 --> 00:36:21,880 when they first did 2017 when they were doing it till now. 559 00:36:22,320 --> 00:36:28,080 And so not only is software going trending towards being 0 560 00:36:28,080 --> 00:36:33,200 price, but the models themselves are trending towards 0. 561 00:36:33,400 --> 00:36:38,560 It is 7 years later, but still man, if you can train it for 562 00:36:38,560 --> 00:36:42,400 under 100 bucks, in seven years from now, you're telling me that 563 00:36:42,400 --> 00:36:46,480 we're going to be able to train Kodak's 5.2 for under 100 bucks? 564 00:36:46,640 --> 00:36:51,480 You have access to a huge amount of intelligence for a very small 565 00:36:51,480 --> 00:36:53,800 price. And when you look at code in 566 00:36:53,800 --> 00:36:58,000 this scenario, it's it's so cheap that you're going to get 567 00:36:58,000 --> 00:37:01,000 software for free. And what happens when we live in 568 00:37:01,000 --> 00:37:07,640 the world that software is free. So my, my personal take is that 569 00:37:07,640 --> 00:37:11,280 it's going to increase a lot of demand for software because it, 570 00:37:11,960 --> 00:37:17,000 it's now easier to build. So we want to build more stuff. 571 00:37:17,400 --> 00:37:22,080 So for example, you said, I'm going to create a small tool 572 00:37:22,080 --> 00:37:25,640 here just to debug a problem that I have or just to 573 00:37:25,640 --> 00:37:29,480 experiment with a different character I want to put in my 574 00:37:29,480 --> 00:37:31,680 game. And and this is not something 575 00:37:31,680 --> 00:37:34,600 that you used to do. Even the game itself. 576 00:37:34,600 --> 00:37:37,880 I'm making a game for my daughter, and I would have never 577 00:37:37,880 --> 00:37:41,880 thought I'm going to code a game so that my daughter can play. 578 00:37:42,200 --> 00:37:45,440 I would have gone to Google and I would have found a game and it 579 00:37:45,880 --> 00:37:48,280 probably would fit a little bit. But it goes back to that 580 00:37:48,280 --> 00:37:54,000 personalization at scale, and now I can have our own game that 581 00:37:54,000 --> 00:37:56,680 I choose the character names, I choose how they look and what 582 00:37:56,680 --> 00:37:59,240 they're doing and what it's related to. 583 00:37:59,240 --> 00:38:01,360 This one specifically around geometry. 584 00:38:01,360 --> 00:38:03,880 And it has horses because my daughter loves that, right? 585 00:38:04,160 --> 00:38:08,600 That's the personalization for me and of one, well, and my 586 00:38:08,600 --> 00:38:09,760 daughter. So I guess I have two. 587 00:38:10,520 --> 00:38:12,280 Yeah. And, and I believe that everyone 588 00:38:12,280 --> 00:38:18,800 will have his or hers 2 belt of programs that you're going to 589 00:38:18,800 --> 00:38:20,960 build and sometimes those will be disposable. 590 00:38:20,960 --> 00:38:24,800 So I create a, a program just to solve this specific case. 591 00:38:25,160 --> 00:38:28,360 And it's so cheap and fast that OK, I can create, I solve my 592 00:38:28,360 --> 00:38:30,640 problem and I'll never touch the software again. 593 00:38:30,760 --> 00:38:33,360 Yeah. Yeah, and you'll want the 594 00:38:33,360 --> 00:38:38,680 novelty, like if I had this horsey game for my daughter, if 595 00:38:38,680 --> 00:38:41,320 it was faster and easier to create. 596 00:38:41,600 --> 00:38:45,080 Because right now I'm spending whatever a weekend on it or a 597 00:38:45,080 --> 00:38:51,040 week. If it was easier to create, then 598 00:38:51,600 --> 00:38:54,680 I would probably make more games I like. 599 00:38:54,680 --> 00:38:57,640 Today I'm playing the horsey game that's revolving around 600 00:38:57,640 --> 00:39:00,760 geometry. Tomorrow I'm going to play the 601 00:39:01,320 --> 00:39:05,920 dog game that revolves around whatever vocabulary expansion or 602 00:39:05,920 --> 00:39:09,160 something like that to try and help my daughter's learning 603 00:39:09,160 --> 00:39:15,080 journey in that regard. And then you augment all of the 604 00:39:15,080 --> 00:39:18,400 different disposable things you're you're creating because 605 00:39:18,400 --> 00:39:22,000 it is so fast and easy to build. Yeah, I, I saw one case recently 606 00:39:22,000 --> 00:39:29,720 that one guy had his MRI scan and there's only one software or 607 00:39:29,720 --> 00:39:33,760 a few software that can do that and it, and it runs on Windows. 608 00:39:33,760 --> 00:39:38,480 And he said, I want to, to see my files and I want to see in my 609 00:39:39,120 --> 00:39:43,280 Mac or Linux or whatever. So he went there to code code 610 00:39:43,280 --> 00:39:49,360 and Vibe coded his MRI file visualizer that runs on the 611 00:39:49,360 --> 00:39:51,440 browser. So this is something crazy. 612 00:39:51,680 --> 00:39:53,680 Yeah, wouldn't happen like a few years ago. 613 00:39:53,680 --> 00:39:57,280 Wow, you at ifood. I'm sure in your current 614 00:39:57,280 --> 00:40:01,600 position you're using a lot of software still that is like 615 00:40:01,680 --> 00:40:03,760 you're and you're probably paying for a lot of software. 616 00:40:05,880 --> 00:40:08,120 Are you going to just stop paying for it? 617 00:40:09,680 --> 00:40:14,560 No, I, I I don't think so. I think when you build a company 618 00:40:14,560 --> 00:40:18,240 around software is not only the code, like you have a lot of 619 00:40:18,240 --> 00:40:21,480 things happening around the code that are very important. 620 00:40:22,520 --> 00:40:26,720 You have the operation, we have support, you have a lot of other 621 00:40:26,720 --> 00:40:31,800 things happening there. And so I, I think this is not 622 00:40:31,800 --> 00:40:35,120 going to happen. It's not going to stop, but 623 00:40:35,200 --> 00:40:39,800 maybe we'll want to build more stuff that previously we 624 00:40:39,800 --> 00:40:43,320 wouldn't like to build. So for example, I have this back 625 00:40:43,320 --> 00:40:47,680 office thing, maybe I'm using external provider today. 626 00:40:47,880 --> 00:40:52,600 But now the cost to build a similar software is so small. 627 00:40:52,600 --> 00:40:56,960 And and if we had a customized, a custom made one would be so 628 00:40:57,440 --> 00:41:01,360 much better for us that now it makes sense to actually put the 629 00:41:01,480 --> 00:41:05,200 the team, the time and the effort to build that because the 630 00:41:05,200 --> 00:41:07,520 cost has decreased. So I think that that could 631 00:41:07,520 --> 00:41:09,800 happen, yeah. Yeah, with the envelopes 632 00:41:09,800 --> 00:41:14,760 community, we were able to save a ton of money because we vibe 633 00:41:14,800 --> 00:41:19,920 coded a newsletter platform. We were spending like, because 634 00:41:19,920 --> 00:41:23,840 once you get to a certain amount of subscribers, you're paying 635 00:41:23,840 --> 00:41:27,080 per subscriber every time you send a newsletter. 636 00:41:27,640 --> 00:41:31,360 And well, each platform has different cost structures, but 637 00:41:31,360 --> 00:41:35,160 the platform we were using, we're paying up the ass for it. 638 00:41:35,560 --> 00:41:40,560 And it was like, well, you know what, maybe we can try and vibe 639 00:41:40,560 --> 00:41:43,840 code something so that we don't have to pay 1000 bucks a month. 640 00:41:44,360 --> 00:41:49,360 And it totally worked. And so now we saved 12 grand. 641 00:41:49,680 --> 00:41:54,000 Yeah. Yeah, the the, the word already 642 00:41:54,000 --> 00:41:57,360 changed like in in, In my opinion, what we're seeing is 643 00:41:57,360 --> 00:42:02,440 that we completely moved the abstraction level that we work 644 00:42:02,440 --> 00:42:05,520 with code. So right now we're not thinking. 645 00:42:05,600 --> 00:42:08,160 It's the same thing that happened when we moved from 646 00:42:08,360 --> 00:42:12,800 assembly to Fortran. For example, this is imagine 647 00:42:13,120 --> 00:42:16,840 John Bacchus arriving and saying, hey, I created this new 648 00:42:16,840 --> 00:42:18,440 programming language. It's really cool. 649 00:42:18,440 --> 00:42:22,200 It's called Fortran and it's easier to program a computer in 650 00:42:22,200 --> 00:42:26,240 that I imagine that people would say, oh, this is, this is not 651 00:42:26,240 --> 00:42:30,600 that good because if I call in assembly, I can choose every 652 00:42:30,600 --> 00:42:34,160 register, I can make every decision, I can choose every 653 00:42:34,160 --> 00:42:36,200 instruction the computer is using. 654 00:42:36,640 --> 00:42:40,200 And and he say, yes, but this is a higher level of abstraction 655 00:42:40,200 --> 00:42:42,160 and so on. And this is something that 656 00:42:42,160 --> 00:42:45,440 completely took the word like right now, most of programming 657 00:42:45,440 --> 00:42:47,640 happens in the high level programming language. 658 00:42:48,160 --> 00:42:52,600 And with those AI tools, what I see is that we're giving another 659 00:42:52,840 --> 00:42:56,400 hop in this abstraction level ladder. 660 00:42:56,720 --> 00:43:00,200 So we're working in a new higher level of abstraction. 661 00:43:00,520 --> 00:43:04,120 It's the it's the kind of the same shift that we had before.