1 00:00:00,240 --> 00:00:03,000 Hi everyone, My name is Patrick Akeel and joining me today is 2 00:00:03,000 --> 00:00:07,120 Sura Hosseini, Co founder over at ORC, where they enable AI 3 00:00:07,120 --> 00:00:09,760 product teams to put AI solutions in production. 4 00:00:10,120 --> 00:00:12,680 And that's exactly what we talked about today, how to be in 5 00:00:12,680 --> 00:00:15,960 control in production with AI solutions with regards to 6 00:00:15,960 --> 00:00:18,560 observability, the specifics of Gen. 7 00:00:18,560 --> 00:00:21,920 AI with regards to models and rate limits and much, much more. 8 00:00:22,160 --> 00:00:24,120 Fascinating conversation in my opinion. 9 00:00:24,120 --> 00:00:30,560 So enjoy from your perspective in this world of AI that we now 10 00:00:30,560 --> 00:00:33,760 live in, things escalated I think a year or two ago, 11 00:00:33,760 --> 00:00:35,160 specifically with regards to Gen. 12 00:00:35,160 --> 00:00:38,040 AI and Chatti BT kind of coming out to the public. 13 00:00:38,360 --> 00:00:43,360 We now have a lot of fluff, hype, potentially also really 14 00:00:43,360 --> 00:00:44,920 good things in production running. 15 00:00:45,160 --> 00:00:49,480 What for you is really valuable in this field of AI and AI 16 00:00:49,480 --> 00:00:52,920 product development and what is still kind of unexplored or more 17 00:00:52,920 --> 00:00:56,800 fluff and hype territory? So what I see a lot of times is 18 00:00:57,040 --> 00:01:00,760 actually I, I will try to answer it in a different way than tell 19 00:01:00,760 --> 00:01:03,800 me if I didn't answer your question right, happy then to 20 00:01:03,880 --> 00:01:08,880 change. But it's a lot of laziness as 21 00:01:08,880 --> 00:01:10,920 well, right? Whenever you're reading up on 22 00:01:10,920 --> 00:01:15,440 stuff or new frameworks come out, new libraries and it always 23 00:01:15,440 --> 00:01:18,520 turns into what this is out with our framework, how you build and 24 00:01:19,400 --> 00:01:22,280 a lead generator or how you build an, a personalized e-mail 25 00:01:22,280 --> 00:01:24,600 outreach. It's always those 2-3 same 26 00:01:24,600 --> 00:01:28,000 examples, which is actually intellectual laziness, right? 27 00:01:28,000 --> 00:01:32,400 And what I'm always trying to sift through is like a very 28 00:01:34,000 --> 00:01:37,840 product specific AI features that teams build and ship that 29 00:01:37,840 --> 00:01:41,640 is so specific to their product and not these boilerplate stuff 30 00:01:41,640 --> 00:01:45,240 like, Oh yeah, sales outreach agent or you know, an e-mail 31 00:01:45,240 --> 00:01:47,640 outreach agent. And everyone has that as an 32 00:01:47,640 --> 00:01:52,800 example. So it's more about what is what 33 00:01:52,800 --> 00:01:55,880 I'm always looking for is in product teams, what is the 34 00:01:56,120 --> 00:02:01,240 strategic differentiator where we can infuse AI into our 35 00:02:01,240 --> 00:02:03,560 product that makes it truly different, right? 36 00:02:03,560 --> 00:02:06,720 Like because the type of data we have or the type of users we 37 00:02:06,720 --> 00:02:11,000 have or the type of workflow we support, what is the creative 38 00:02:11,480 --> 00:02:14,960 rethinking we can do around that to infuse that with AI? 39 00:02:16,760 --> 00:02:20,080 That gets me going and excited. And whenever I read, oh, this is 40 00:02:20,080 --> 00:02:24,280 how we do ICP outreach or, you know, lead generation outreach. 41 00:02:24,800 --> 00:02:29,280 Yeah, then that's kind of a, a fluff for me. 42 00:02:29,640 --> 00:02:32,640 And another one is, you know, all these people on LinkedIn 43 00:02:33,000 --> 00:02:36,920 writing stuff and then you open their profile and you see like 44 00:02:36,920 --> 00:02:40,000 they're now an AI strategist, but strategist. 45 00:02:40,000 --> 00:02:42,600 But six months ago, they were still like 12 years of project 46 00:02:42,600 --> 00:02:45,360 manager and some boring bank, and now they're an AI 47 00:02:45,360 --> 00:02:49,800 strategist. That's also like, yeah, is 48 00:02:49,800 --> 00:02:52,440 someone actually building and shipping or are they just 49 00:02:52,440 --> 00:02:55,800 regurgitating stuff that they found or they have just checked 50 00:02:55,800 --> 00:03:00,000 GPT, write some blogs for them to post on LinkedIn to become an 51 00:03:00,000 --> 00:03:01,920 influencer. Yeah. 52 00:03:01,920 --> 00:03:04,400 So it's either do you have the track record or are you doing 53 00:03:04,400 --> 00:03:07,680 something really creative with AI, right? 54 00:03:08,680 --> 00:03:12,400 Molding it in your favour instead of just the standard 55 00:03:12,920 --> 00:03:14,320 right to personalized e-mail outreach. 56 00:03:14,320 --> 00:03:16,200 Because my inbox is full of them, right? 57 00:03:16,200 --> 00:03:20,320 Like my spam box at this at any given day is like around 150 58 00:03:20,640 --> 00:03:24,960 called emails a day with yeah these AI generated crap that no 59 00:03:24,960 --> 00:03:27,240 one reads. It's a, it's a sad reality that 60 00:03:27,280 --> 00:03:30,440 we get there right. And I, I understand from let's 61 00:03:30,440 --> 00:03:34,080 say the perspective of someone that sees this out on the market 62 00:03:34,080 --> 00:03:36,960 that really wants to actually contribute, how do you 63 00:03:36,960 --> 00:03:39,160 differentiate yourself? That's probably a question 64 00:03:39,160 --> 00:03:42,040 they've asked themselves, but the answer is the same, right? 65 00:03:42,080 --> 00:03:43,920 Yeah. Act like you're an expert, or 66 00:03:43,920 --> 00:03:46,760 start creating content or start playing around with it and 67 00:03:46,760 --> 00:03:48,480 actually see how you can deliver value. 68 00:03:48,800 --> 00:03:52,480 The easiest part is to act the part and not actually do the 69 00:03:52,480 --> 00:03:54,080 thing. Do the work, put in the effort 70 00:03:54,080 --> 00:03:55,880 to try and understand deliver value. 71 00:03:56,120 --> 00:03:58,880 Figure out use cases in your personal productivity, and then 72 00:03:58,880 --> 00:04:01,840 see what use cases can be solved in in the outside world. 73 00:04:02,720 --> 00:04:04,280 So maybe people take the easy Rd. 74 00:04:04,800 --> 00:04:08,360 And maybe just on what's fluff and what's not fluff, right? 75 00:04:08,360 --> 00:04:11,400 So are people actually building and shipping or just talking 76 00:04:11,400 --> 00:04:13,320 about it, right? It's easy to filter those out. 77 00:04:13,880 --> 00:04:18,640 But also important one is whoever thinks AI is this magic 78 00:04:18,959 --> 00:04:21,200 pill you take and all of a sudden your life changes, right? 79 00:04:21,320 --> 00:04:25,040 Whoever makes that promise in this current day and age, right? 80 00:04:25,040 --> 00:04:27,800 I'm not saying five years down the road what AI will be able to 81 00:04:27,800 --> 00:04:30,200 do, but now it's never a magic pill. 82 00:04:30,400 --> 00:04:32,680 Like you just turn it on and you're all you start getting 83 00:04:32,680 --> 00:04:35,960 leads and seals and you know, everything blows up. 84 00:04:37,200 --> 00:04:40,360 So you see that also often right where it's being sold as just 85 00:04:40,360 --> 00:04:43,600 take this magic pill and your whole life will be better after 86 00:04:43,600 --> 00:04:46,720 this. Just turn our on our AI sales 87 00:04:46,720 --> 00:04:50,000 agent and your whole funnel will be filled up with qualified 88 00:04:50,000 --> 00:04:50,720 leads. Yeah. 89 00:04:51,000 --> 00:04:53,440 So that's also important to see us fluff I. 90 00:04:53,520 --> 00:04:55,240 Feel like. Same people saying I'll take 91 00:04:55,240 --> 00:04:56,520 this pill and you'll lose weight. 92 00:04:56,560 --> 00:04:58,120 Yeah, it's never the case, right? 93 00:04:58,120 --> 00:05:01,440 It's a it's a grind to really do something. 94 00:05:01,520 --> 00:05:02,960 I mean, it's always been a thing. 95 00:05:02,960 --> 00:05:05,040 The the example that you give, take this pill and you'll lose 96 00:05:05,040 --> 00:05:06,440 weight. Like that's what people want. 97 00:05:06,920 --> 00:05:08,160 Yeah. And that now is kind of out 98 00:05:08,160 --> 00:05:10,120 there with Ozempic. But we won't go in there like 99 00:05:10,120 --> 00:05:11,520 that. That's what people are looking 100 00:05:11,520 --> 00:05:12,480 for. And when they're there, they 101 00:05:12,480 --> 00:05:14,000 will take it. That's the easiest way. 102 00:05:14,040 --> 00:05:16,600 Well, even with an Ozempic, you need to change your lifestyle. 103 00:05:16,600 --> 00:05:19,000 Yeah, you need to work out, you need to do strength training. 104 00:05:19,000 --> 00:05:22,320 So it's not like, oh, I'll take a shot and my life changes, 105 00:05:22,320 --> 00:05:24,880 right? So yeah, yeah, it's really 106 00:05:25,920 --> 00:05:30,080 experimenting, testing, making mistakes, right? 107 00:05:30,080 --> 00:05:32,760 It's almost like building a startup, like getting your 108 00:05:32,760 --> 00:05:36,640 organization going with AI. It's almost as building with a 109 00:05:36,640 --> 00:05:39,440 startup. You have idea, you test it, it 110 00:05:39,440 --> 00:05:41,840 works, it doesn't work. You roll it back, you change it, 111 00:05:42,160 --> 00:05:46,840 right. So yeah, it's never a magic pill 112 00:05:46,840 --> 00:05:50,000 for anyone, I think. Have you seen any common use 113 00:05:50,000 --> 00:05:53,160 cases that have been put into production with regards to AI? 114 00:05:53,160 --> 00:05:55,360 Stuff that actually works, even though the data might be 115 00:05:55,360 --> 00:05:59,360 specific to an industry or an organization by using the tools 116 00:05:59,360 --> 00:06:01,240 in this way, like. That's a solid use case. 117 00:06:01,440 --> 00:06:03,760 Yeah, already right. A couple have been proven, which 118 00:06:03,760 --> 00:06:07,680 I also kind of believe so one is of course code, right, coding. 119 00:06:08,080 --> 00:06:12,000 We at this moment, right, like all our engineers have to be AI 120 00:06:12,000 --> 00:06:14,040 first. So we don't hire juniors 121 00:06:14,040 --> 00:06:17,080 anymore. It's either matured Meteors or 122 00:06:17,080 --> 00:06:19,560 definitely seniors because at any given time they're 123 00:06:19,560 --> 00:06:24,280 orchestrating, you know, four or five coding agents that actually 124 00:06:24,280 --> 00:06:26,920 output high quality work. And our platform is really, 125 00:06:26,920 --> 00:06:30,240 really complex. So it's not like, oh, just, you 126 00:06:30,240 --> 00:06:33,600 know, create a landing page, but it still does a really good job 127 00:06:33,600 --> 00:06:36,080 and it improves our efficiency, right? 128 00:06:36,080 --> 00:06:38,800 So coding is already clear that that works. 129 00:06:39,880 --> 00:06:43,560 Customer support, right? That's clearly one that works. 130 00:06:44,240 --> 00:06:47,160 It's also why those are always the examples that are being used 131 00:06:47,160 --> 00:06:50,000 because those are more matured and have proven that they 132 00:06:50,000 --> 00:06:52,600 actually work. Yeah. 133 00:06:52,600 --> 00:06:55,600 So like in big buckets, those are the two buckets that already 134 00:06:55,600 --> 00:06:58,600 prove value if implemented correctly. 135 00:06:58,680 --> 00:07:00,440 Right. So with coding, your people need 136 00:07:00,440 --> 00:07:03,800 to know what they're doing. The orchestrators of the agents 137 00:07:04,280 --> 00:07:08,600 need to know the architecture, the vision of the product so 138 00:07:08,600 --> 00:07:11,120 they can assess the quality of the output, right. 139 00:07:11,120 --> 00:07:12,840 So you can't. Again, it's not a magic pill. 140 00:07:13,000 --> 00:07:14,960 You. I cannot as a non-technical 141 00:07:14,960 --> 00:07:19,120 founder, turn on five coding agents and then develop a 142 00:07:19,120 --> 00:07:21,080 business critical scalable platform. 143 00:07:21,520 --> 00:07:24,080 So the senior people orchestrating them need to know 144 00:07:24,080 --> 00:07:26,200 what they're doing. It's just that they now all of a 145 00:07:26,200 --> 00:07:30,600 sudden have these tireless team members that can output the, the 146 00:07:30,600 --> 00:07:34,440 work and they are the quality assurance for the architecture 147 00:07:34,440 --> 00:07:38,080 and scalability of the platform. Same with customer support. 148 00:07:39,160 --> 00:07:42,600 It's not just, oh, we'll upload all our FAQs in a, in a chat bot 149 00:07:42,600 --> 00:07:46,320 and then it's solved, right, Because you will have knowledge 150 00:07:46,320 --> 00:07:47,640 gaps. The data is outdated. 151 00:07:47,640 --> 00:07:51,440 So you are, you do need to change your processes and change 152 00:07:51,440 --> 00:07:54,040 some roles of people and make other people responsible for 153 00:07:54,040 --> 00:07:56,760 maintaining the knowledge gaps. But how do you have visibility 154 00:07:56,760 --> 00:07:58,600 on the knowledge gaps of your AI chat bot? 155 00:07:58,920 --> 00:08:02,880 And do you give the AI chat bot support bot access to different 156 00:08:02,880 --> 00:08:08,120 APIs so they can do a refund or, you know, do some trigger some 157 00:08:08,120 --> 00:08:12,080 specific processes? Yeah, all that you need to 158 00:08:12,080 --> 00:08:14,160 solve, right. Again, there's no magic pill 159 00:08:14,160 --> 00:08:17,400 that then does that for you. Yeah, I have seen people and 160 00:08:17,400 --> 00:08:20,480 especially now that MCP came out as kind of this new integration 161 00:08:20,480 --> 00:08:24,240 block that are afraid of doing that right, giving a model then 162 00:08:24,240 --> 00:08:27,520 access to refunding. And like I, I get it from an 163 00:08:27,520 --> 00:08:29,000 information gathering standpoint. 164 00:08:29,000 --> 00:08:32,640 If I'm having a production issue and I have a multitude of 165 00:08:32,640 --> 00:08:35,240 systems, I would love to have one interface that goes to all 166 00:08:35,240 --> 00:08:37,760 the systems and it gives me a diagnosis. 167 00:08:37,760 --> 00:08:39,039 Yeah. And I can go back and forth with 168 00:08:39,039 --> 00:08:42,000 regards to one interface that just gets the information from 169 00:08:42,000 --> 00:08:43,799 all systems, but I'm in the driver's seat. 170 00:08:43,799 --> 00:08:46,360 Yeah. If I were to then be like, yeah, 171 00:08:46,360 --> 00:08:49,000 I'm a customer support engineer, and I can see issues with 172 00:08:49,000 --> 00:08:51,560 regards to an order and then an automatic refund kicks in. 173 00:08:51,560 --> 00:08:54,280 And I was like, maybe that's not the right solution, but 174 00:08:54,320 --> 00:08:55,920 something else decided to do that. 175 00:08:56,200 --> 00:08:57,720 That's where people are a bit more fearful. 176 00:08:57,720 --> 00:09:01,360 No. And that's it's all about 177 00:09:01,360 --> 00:09:04,200 control, right? Do are you in control? 178 00:09:04,880 --> 00:09:08,440 Control of the costs of the performance of the data, of the 179 00:09:08,440 --> 00:09:11,880 actions that the AI is doing. So how do you give someone a 180 00:09:11,880 --> 00:09:13,320 sense of control? Right. 181 00:09:13,320 --> 00:09:16,360 The same way that a dashboard in a car has been invented over 182 00:09:16,360 --> 00:09:19,280 time because you as a driver, it is proven. 183 00:09:19,280 --> 00:09:21,960 OK, I need to have understand how fast am I going? 184 00:09:22,160 --> 00:09:24,600 How's my engine doing? Do I have my lights on or not? 185 00:09:24,600 --> 00:09:26,040 Right. There's some information you 186 00:09:26,040 --> 00:09:27,800 need. And that's why a dashboard has 187 00:09:27,800 --> 00:09:29,800 evolved into what it is. Yeah. 188 00:09:30,120 --> 00:09:34,080 We're now in this change. Where people start putting AI in 189 00:09:34,080 --> 00:09:36,840 production and then they realize, oh, I don't have 190 00:09:36,840 --> 00:09:39,520 control, right? Or I'm not in control or this 191 00:09:39,520 --> 00:09:42,960 use case sounds cool, right? Giving refunds, but I need to be 192 00:09:42,960 --> 00:09:45,080 in control. We need to have auditability. 193 00:09:45,320 --> 00:09:47,040 There needs to be maybe a human in the loop. 194 00:09:47,280 --> 00:09:49,520 But if the human is in the loop, the human needs to have the 195 00:09:49,520 --> 00:09:52,040 information to say yes or no, right? 196 00:09:52,040 --> 00:09:53,880 Can I just say, hey, can I make a refund? 197 00:09:54,080 --> 00:09:55,880 Because seven questions will come after that. 198 00:09:55,880 --> 00:09:58,160 Like to for what? What's the reasoning, right? 199 00:09:58,440 --> 00:10:01,000 So here's that. Here's my information why I 200 00:10:01,000 --> 00:10:02,720 think this person needs a refund. 201 00:10:03,120 --> 00:10:03,960 Do you think? Right. 202 00:10:04,360 --> 00:10:08,400 So an entirely new what we call agent control tower, but it's 203 00:10:08,400 --> 00:10:12,800 like human agent interface now needs to be invented, right? 204 00:10:13,120 --> 00:10:17,280 How does that look like air traffic control tower in a, in 205 00:10:17,280 --> 00:10:21,600 a, in a airport, right? Like, how does the air traffic 206 00:10:21,600 --> 00:10:25,680 controller feel in power, in control, that they're not going 207 00:10:25,680 --> 00:10:27,360 to make mistakes and create accidents? 208 00:10:27,600 --> 00:10:30,680 Well, that is now organizations are going to need these agent 209 00:10:30,680 --> 00:10:35,080 control towers where an individual can orchestrate a 210 00:10:35,080 --> 00:10:38,200 swarm of agents. And it's different per use case, 211 00:10:38,200 --> 00:10:39,760 right? But it's like the agents are 212 00:10:39,760 --> 00:10:41,280 going to get stuck. They're going to throw 213 00:10:41,280 --> 00:10:43,760 exceptions, they're going to ask for approvals. 214 00:10:44,160 --> 00:10:45,880 But how do you feed that to the human? 215 00:10:45,960 --> 00:10:48,000 So first of all, they're not flooded with the information 216 00:10:48,000 --> 00:10:50,680 because they cannot read 20,000 logs per second. 217 00:10:52,280 --> 00:10:56,480 And yeah, so you have the just in time and management by 218 00:10:56,480 --> 00:10:59,520 exception principles, right. Everything has already been 219 00:10:59,960 --> 00:11:03,400 invented in, in management and business administration is like, 220 00:11:03,960 --> 00:11:07,680 what's the just in time flow you need? 221 00:11:07,680 --> 00:11:09,960 What is the management by exception flows you need? 222 00:11:10,360 --> 00:11:12,440 So an orchestrator can do their work. 223 00:11:14,280 --> 00:11:18,720 Yeah, that's now the phase we're in where now the the next stage 224 00:11:18,720 --> 00:11:21,520 is starting to become uncomfortable for people, 225 00:11:21,520 --> 00:11:24,200 forcing us to invent new concepts. 226 00:11:24,200 --> 00:11:27,640 Yeah, I want to get into like the maturity of organizations. 227 00:11:27,640 --> 00:11:30,920 What maturity you want to have as an organization before you go 228 00:11:30,920 --> 00:11:32,680 there. But before we do, let's touch on 229 00:11:32,680 --> 00:11:34,520 a little bit of what you're doing with ORC, because I think 230 00:11:34,520 --> 00:11:36,240 we already hit some of the surface there. 231 00:11:36,520 --> 00:11:40,560 Oh sure. So at ORC we we provide an AI 232 00:11:40,560 --> 00:11:44,080 engineering platform. So it's a SAS that we build it, 233 00:11:44,080 --> 00:11:48,520 we maintain it for teams where engineering teams and product 234 00:11:48,520 --> 00:11:53,320 teams who have a vision, right, who who want to infuse either 235 00:11:53,320 --> 00:11:57,880 existing or new products with generative AI in a scalable way. 236 00:11:58,040 --> 00:12:02,680 So they use our platform and middleware in there to develop 237 00:12:02,680 --> 00:12:07,200 hypotheses, test and experiment with their hypotheses, evaluate 238 00:12:07,680 --> 00:12:11,120 how their use cases are going to perform next, put it in 239 00:12:11,120 --> 00:12:14,520 production, but in production to do granular rollouts, Canary 240 00:12:14,520 --> 00:12:19,800 releases, AB testing. So that's you know what whatever 241 00:12:19,840 --> 00:12:24,040 AI feature you have shipped in production acts and works the 242 00:12:24,040 --> 00:12:27,840 way you intended it again, control and then observability 243 00:12:27,840 --> 00:12:29,840 monitoring evaluations needed there. 244 00:12:29,840 --> 00:12:32,400 And then finally the human in the loop, right. 245 00:12:32,400 --> 00:12:36,040 So the product managers, domain experts, right? 246 00:12:36,120 --> 00:12:38,480 They're not all Python engineers. 247 00:12:39,280 --> 00:12:42,240 How can they be involved in this whole process to give feedback, 248 00:12:42,240 --> 00:12:45,080 to make corrections, to identify hallucinations? 249 00:12:45,880 --> 00:12:49,200 Now teams are patching this together, either with some 250 00:12:49,200 --> 00:12:54,040 random open source libraries or they build their own solutions. 251 00:12:54,760 --> 00:12:58,520 Well, we bring this as a fully managed, well integrated 252 00:12:58,680 --> 00:13:02,360 platform. So the teams can just focus on 253 00:13:02,360 --> 00:13:04,960 building their features instead of maintaining this whole 254 00:13:05,360 --> 00:13:07,080 enablement layer underneath. Yeah. 255 00:13:07,400 --> 00:13:10,480 Because one thing teams never have enough of is resources, 256 00:13:10,480 --> 00:13:12,480 right? So the whoever they can get on 257 00:13:12,480 --> 00:13:15,640 board and the right AI engineers, they can find, you 258 00:13:15,640 --> 00:13:19,080 want them to be working on your features and not maintaining, 259 00:13:19,400 --> 00:13:20,960 let's say, the pipelines under the ground. 260 00:13:21,200 --> 00:13:22,800 So that's where we then come in. Yeah. 261 00:13:23,360 --> 00:13:26,880 And then finally, because of our AI gauge where we're integrated 262 00:13:26,880 --> 00:13:29,600 with all the hyper scalers, all the cloud providers, all the 263 00:13:29,600 --> 00:13:33,120 model providers and we unify that into a single API. 264 00:13:33,640 --> 00:13:36,600 So teams first of all are not locked into a single provider 265 00:13:36,840 --> 00:13:40,440 and they can actually operate multi cloud out-of-the-box. 266 00:13:40,960 --> 00:13:43,840 And and now with all the geopolitical stuff and data 267 00:13:43,840 --> 00:13:47,600 residency concerns, they can even decide like which providers 268 00:13:47,600 --> 00:13:51,240 do we use for which entity, in which region, etcetera. 269 00:13:52,160 --> 00:13:54,640 Again, control, control, control that they need. 270 00:13:54,840 --> 00:13:56,680 Yeah. I think that's mainly like if I 271 00:13:56,680 --> 00:13:59,880 look at software development life cycle, we have strong 272 00:13:59,880 --> 00:14:02,480 conventions and those conventions can still differ in 273 00:14:02,480 --> 00:14:06,640 organizations, but the tooling is there to kind of adhere to 274 00:14:06,640 --> 00:14:09,680 those conventions to put them in place or we don't reinvent the 275 00:14:09,680 --> 00:14:12,400 wheel there necessarily. And from what I hear you're 276 00:14:12,400 --> 00:14:14,800 saying, you're trying to be one of the tools in the tool belt 277 00:14:14,800 --> 00:14:16,920 there. And a lot of people are trying 278 00:14:16,920 --> 00:14:19,640 to figure out, do I build my own or do I indeed grab something 279 00:14:19,640 --> 00:14:21,080 off the shelf, which I think validly. 280 00:14:21,080 --> 00:14:24,120 So with Gen. AI in production, we've had I 281 00:14:24,120 --> 00:14:26,840 think the most out of control situations in production, right? 282 00:14:27,240 --> 00:14:29,680 Because I've had many demos where people tell me, this is 283 00:14:29,680 --> 00:14:31,960 our product. We do XY and Z and then 284 00:14:31,960 --> 00:14:33,360 something happens and they're like, well, it's still 285 00:14:33,360 --> 00:14:34,920 generative AI. So there is a level of 286 00:14:34,920 --> 00:14:36,680 hallucination. That's not what I want to hear, 287 00:14:36,840 --> 00:14:39,160 especially if I'm buying a product like I want, I want 288 00:14:39,160 --> 00:14:41,360 guarantees. I do want that level of control. 289 00:14:41,600 --> 00:14:43,760 I don't want someone to say, well, it's a thing and it just 290 00:14:43,760 --> 00:14:46,360 hallucinates, so I fully understand that people are 291 00:14:46,360 --> 00:14:47,720 looking for that. Yeah, of course. 292 00:14:48,240 --> 00:14:50,520 You know, you want to make the margin of error as small as 293 00:14:50,520 --> 00:14:52,480 possible. Yeah, but if you had humans 294 00:14:52,480 --> 00:14:56,160 doing that job, you would not be shocked if one of the employees 295 00:14:56,160 --> 00:14:58,160 would make a mistake, right? You would not hold them 296 00:14:58,160 --> 00:15:02,440 accountable for 100% perfection. Yeah, most of the time depends 297 00:15:02,440 --> 00:15:05,240 on which field, but most fields you know, you're not going to 298 00:15:05,240 --> 00:15:09,480 fire your people if they make, you know, one in 1000 errors, 299 00:15:09,480 --> 00:15:11,880 right. But you, you do have that kind 300 00:15:11,880 --> 00:15:15,040 of tendency to when it comes to AI or self driving cars as well, 301 00:15:15,040 --> 00:15:17,360 right? Like how many people are getting 302 00:15:17,360 --> 00:15:19,720 killed by normal cars versus self driving cars. 303 00:15:19,720 --> 00:15:21,680 But still, right, 1 is already too many. 304 00:15:22,440 --> 00:15:24,800 So that's kind of also a mind shift that needs to happen. 305 00:15:24,880 --> 00:15:29,280 But indeed, in in software delivery, as you mentioned, a 306 00:15:29,280 --> 00:15:33,560 lot of the concepts are, you know, decided on, right. 307 00:15:33,560 --> 00:15:36,360 You need code control, you need pipelines, you need CICD, you 308 00:15:36,360 --> 00:15:39,920 need a regression testing. No one questions that. 309 00:15:40,440 --> 00:15:42,800 And then either even some of those you buy or build. 310 00:15:43,360 --> 00:15:46,040 But now you're starting adding these little black boxes of 311 00:15:46,160 --> 00:15:48,320 unpredictability in your software. 312 00:15:48,360 --> 00:15:50,920 Yeah, right. That behind the scenes even get 313 00:15:50,920 --> 00:15:52,520 updated without you knowing it, right. 314 00:15:52,520 --> 00:15:56,360 Like open AI behind and they can just update their model behind 315 00:15:56,360 --> 00:15:58,200 an API and you would not even know it. 316 00:15:58,880 --> 00:16:01,200 And all of a sudden your use case starts acting differently. 317 00:16:01,720 --> 00:16:05,560 So the life cycle management, which is now not being really 318 00:16:05,560 --> 00:16:09,040 discussed becomes a problem. Everyone is already happy if 319 00:16:09,040 --> 00:16:12,120 they get Apoc out the door that they can demo to management and 320 00:16:12,120 --> 00:16:14,480 get some applauses. But no one has thought about, 321 00:16:14,480 --> 00:16:17,720 OK, well, how do we put it in production and how do we get 322 00:16:17,720 --> 00:16:20,440 into then the continuous delivery and improvement flow, 323 00:16:20,680 --> 00:16:22,440 right. You also would as a with your 324 00:16:22,440 --> 00:16:24,840 background in the product management, etcetera, you know, 325 00:16:24,840 --> 00:16:26,960 like it's getting it out there the first time. 326 00:16:26,960 --> 00:16:29,520 That's when the the work actually starts because then you 327 00:16:29,520 --> 00:16:32,120 start getting feedback and learnings and edge cases and 328 00:16:32,120 --> 00:16:35,320 weird behaviors. So how do you set yourself up 329 00:16:35,320 --> 00:16:38,240 for the cycles of learning insights, changing your 330 00:16:38,240 --> 00:16:40,600 hypothesis, testing them, putting it on my back in 331 00:16:40,600 --> 00:16:42,920 production? And now what you see with teams 332 00:16:42,920 --> 00:16:44,480 is then they get totally stuck, right? 333 00:16:44,480 --> 00:16:47,120 Everyone start trembling over each other and tripping and 334 00:16:47,400 --> 00:16:50,680 Excel sheets with the color-coded corrections getting 335 00:16:50,680 --> 00:16:54,800 emailed back and forward, the engineers going crazy because 336 00:16:55,800 --> 00:16:58,640 it's not, you know, solidified yet the process. 337 00:16:58,640 --> 00:17:02,440 So that's also where we try to come in is giving you that end 338 00:17:02,440 --> 00:17:05,920 to end process. So you can do this continuous 339 00:17:05,920 --> 00:17:08,319 delivery almost like CICD for Gen. 340 00:17:08,319 --> 00:17:09,520 AI right? Yeah. 341 00:17:09,520 --> 00:17:12,800 Is that then what the a resilient production ready 342 00:17:12,800 --> 00:17:15,200 solution looks like with regards to an in control state. 343 00:17:15,520 --> 00:17:19,359 So I deploy something I have insights with regards to, is it 344 00:17:19,359 --> 00:17:21,800 guard reels, is it boundaries, is it performance? 345 00:17:21,800 --> 00:17:22,880 Yeah, everything with regards to. 346 00:17:22,880 --> 00:17:26,920 Observability definitely. I mean again related back to 347 00:17:26,920 --> 00:17:30,360 normal software, right? You will check how fast is it 348 00:17:30,360 --> 00:17:33,720 working, what's the latency? Why is this specific call taking 349 00:17:33,720 --> 00:17:34,760 so long? Oh, I don't know. 350 00:17:34,760 --> 00:17:37,360 We have some, you know, we don't have an index in the database 351 00:17:37,360 --> 00:17:39,160 for this one. That's why it's so slow. 352 00:17:39,160 --> 00:17:42,400 We need to improve it. You will have the same analogous 353 00:17:42,400 --> 00:17:43,920 challenges with Gen. AI, right? 354 00:17:43,920 --> 00:17:45,800 Like why is this model getting slow over time? 355 00:17:46,360 --> 00:17:48,040 Why are we hitting our rate limits? 356 00:17:48,640 --> 00:17:51,840 Do we need to do and do orchestration, retries, 357 00:17:51,840 --> 00:17:55,240 fallbacks? The models are not up 99.999%. 358 00:17:55,600 --> 00:17:59,800 So how do you build in resiliency Cost is a compared to 359 00:17:59,800 --> 00:18:03,320 normal traditional AP is it's it's very different than Gen. 360 00:18:03,320 --> 00:18:05,360 AI, right? You can literally get yourself 361 00:18:05,360 --> 00:18:09,800 into negative margins by building the wrong AI feature. 362 00:18:09,880 --> 00:18:16,440 That was one of we were speaking to a prospect who they do it 363 00:18:16,440 --> 00:18:19,920 like a billion revenue a year and they were considering to and 364 00:18:19,920 --> 00:18:22,800 they build APOC of a feature that does some really cool 365 00:18:22,800 --> 00:18:25,960 recommendations. But he was like, if we would 366 00:18:25,960 --> 00:18:31,720 turn this on in production and half of our users would actually 367 00:18:31,720 --> 00:18:34,840 use it, Yeah. Then we would spend 500 million 368 00:18:34,840 --> 00:18:38,960 a year on AI tokens, which is half of your revenue. 369 00:18:38,960 --> 00:18:41,600 Yeah, just on that one feature, which is not even a killer 370 00:18:41,600 --> 00:18:45,600 feature that will change your life as a either user or as a 371 00:18:45,600 --> 00:18:47,880 business. But it was recommendation half 372 00:18:47,880 --> 00:18:51,200 of their revenue, right. So cost is, for example, 373 00:18:51,200 --> 00:18:53,440 something that's really important and who is responsible 374 00:18:53,440 --> 00:18:55,120 for cost? In a lot of matured 375 00:18:55,120 --> 00:18:57,000 organizations, the product manager, right? 376 00:18:57,000 --> 00:18:59,560 They own the PNL of their product or their features. 377 00:18:59,800 --> 00:19:02,400 So they need to know, hey, how much is it costing me to ship 378 00:19:02,400 --> 00:19:05,160 this feature versus what is it getting us? 379 00:19:05,640 --> 00:19:10,480 So again, control right from multiple dimensions you need. 380 00:19:10,760 --> 00:19:13,120 Interesting. Yeah, I feel like a lot of 381 00:19:13,120 --> 00:19:17,560 people are seeing where they can deliver value indeed. 382 00:19:17,840 --> 00:19:20,240 But then if it's worth it, that's a whole another question, 383 00:19:20,360 --> 00:19:22,440 right? I've seen many proof of concepts 384 00:19:22,440 --> 00:19:25,000 with regards to, I mean, the environment that I used to be 385 00:19:25,000 --> 00:19:28,640 in, which is a bank because it's a really cool technology and we 386 00:19:28,640 --> 00:19:30,320 don't want to be behind. So we're going to do proof of 387 00:19:30,320 --> 00:19:32,240 concepts to see where we can deliver value. 388 00:19:32,240 --> 00:19:35,320 And if it's a yes, then usually we should look into scaling 389 00:19:35,320 --> 00:19:37,240 things. But then through that you also 390 00:19:37,240 --> 00:19:40,280 can congest the system because only a few things can have 391 00:19:40,520 --> 00:19:42,680 support from the right engineers and actually make it to 392 00:19:42,680 --> 00:19:45,400 production. And if you just go wide with 393 00:19:45,400 --> 00:19:47,520 regards to use cases, you don't actually see it. 394 00:19:47,520 --> 00:19:50,120 OK, everything has value, but what is the cost and what is the 395 00:19:50,120 --> 00:19:52,280 business case? In the end of the day, I feel 396 00:19:52,280 --> 00:19:56,520 like there is a lot of value in reducing manual labour where you 397 00:19:56,520 --> 00:20:00,400 just have documents, PDFs and people are going from that to 398 00:20:00,400 --> 00:20:02,120 filling it in. In the systems I've seen 399 00:20:02,120 --> 00:20:05,880 hundreds of pages. Manually analysed to be then put 400 00:20:05,880 --> 00:20:08,760 into a system. I feel like there is like a very 401 00:20:08,760 --> 00:20:11,040 big use case. The sooner you can have those 402 00:20:11,040 --> 00:20:13,080 data points kind of automatically pre filled it 403 00:20:13,320 --> 00:20:17,040 human in the loop with regards to kind of an assessment for I 404 00:20:17,040 --> 00:20:20,120 principle as you will. If the first two is are AI and 405 00:20:20,120 --> 00:20:22,520 then go through like you can increase productivity in that 406 00:20:22,520 --> 00:20:26,400 way and that is more of a cost reduction exercise. 407 00:20:26,400 --> 00:20:29,040 But for me, that's a very tangible use case. 408 00:20:29,440 --> 00:20:32,680 Yes, and at the same time, but that's like my strategy 409 00:20:32,680 --> 00:20:36,760 consultant had on immediately will be like, but was that 410 00:20:36,760 --> 00:20:38,960 process needed in the 1st place, right. 411 00:20:38,960 --> 00:20:43,120 So sometimes even just blank campus, blank slate, rethinking 412 00:20:43,120 --> 00:20:45,960 the whole process of why, why, how did we end up in this 413 00:20:45,960 --> 00:20:50,680 situation with this workflow and then see what we can do with AI 414 00:20:50,680 --> 00:20:54,200 instead of just trying to patch it with AI Because yeah, you're 415 00:20:54,200 --> 00:20:56,760 just trying to fix a shitty process to begin with, right? 416 00:20:56,760 --> 00:21:02,960 Like, like we're now in going through some funding processes 417 00:21:02,960 --> 00:21:06,560 at the moment and then I'm starting being confronted and 418 00:21:06,560 --> 00:21:09,720 I'm super allergic by all these inefficient processes, right? 419 00:21:09,720 --> 00:21:15,520 Like we literally, I have to drive to one note tree to pick 420 00:21:15,520 --> 00:21:18,520 up a physical book, yeah, of the shareholders and then drive it 421 00:21:18,520 --> 00:21:20,880 to a different note tree and they need that booklet. 422 00:21:22,040 --> 00:21:25,000 We need to go everywhere with our IDs, my Co, Vander and me to 423 00:21:25,200 --> 00:21:29,400 identify ourselves in person even though they have everything 424 00:21:29,400 --> 00:21:32,840 about us already. Endless amount of KYC forms I 425 00:21:32,840 --> 00:21:34,480 need to fill in. Even though the Chamber of 426 00:21:34,480 --> 00:21:37,560 Commerce has all that information, like I'm registered 427 00:21:37,560 --> 00:21:40,360 as the, you know, the director. But then I'd still need to 428 00:21:40,360 --> 00:21:43,240 manually fill in some random PDF with all those. 429 00:21:43,640 --> 00:21:47,240 So it's like, OK, I can automate that with AI, but isn't it a 430 00:21:47,240 --> 00:21:50,840 stupid process to begin with? I like you as a no tree. 431 00:21:50,960 --> 00:21:53,680 Why don't you just pull that from the Chamber of Commerce? 432 00:21:53,680 --> 00:21:56,520 Why do you want me to download it from the Chamber of Commerce 433 00:21:56,840 --> 00:22:00,760 and print it out and then fill it in by hand and then post it 434 00:22:00,760 --> 00:22:01,320 to you? Right. 435 00:22:01,320 --> 00:22:03,560 It's like the whole process sucks to begin with. 436 00:22:03,800 --> 00:22:07,480 Yeah, I mean, if people come to ORC then specifically or to you 437 00:22:07,480 --> 00:22:10,200 with regards to a use case, and we want to use ORC in production 438 00:22:10,200 --> 00:22:12,800 because we think we can be in control, Do you then also 439 00:22:12,800 --> 00:22:15,960 challenge those use cases? Because I think from your, if I 440 00:22:15,960 --> 00:22:18,440 were your customer, for me that'd be incredibly valuable, 441 00:22:18,440 --> 00:22:20,800 right? AII see this, my competitors are 442 00:22:20,800 --> 00:22:22,240 doing this. I have the feeling I'm behind. 443 00:22:22,240 --> 00:22:24,120 I want to do something resilient in production. 444 00:22:24,360 --> 00:22:26,400 Think you're a great fit. This is our use case. 445 00:22:26,680 --> 00:22:29,480 And if you then say, well, actually do you need to have 446 00:22:29,480 --> 00:22:31,280 that in the 1st place, that would be incredibly valuable, 447 00:22:31,320 --> 00:22:37,440 yeah. We do, although we have great 448 00:22:37,680 --> 00:22:41,680 consultancy and strategy partners who do that in the more 449 00:22:41,680 --> 00:22:45,560 complex projects already. So they go and do that. 450 00:22:45,560 --> 00:22:48,840 So we come in with the technology after the second is 451 00:22:51,200 --> 00:22:54,920 who are not our buyers are teams that are thinking about starting 452 00:22:54,920 --> 00:22:56,240 with Gen. AI, right? 453 00:22:56,240 --> 00:22:59,680 Because then we're trying to sell them a Formula One car when 454 00:22:59,680 --> 00:23:02,920 they don't even know if they need a bike or, or, or a scooter 455 00:23:02,920 --> 00:23:06,720 or a car. So the teams that knock on our 456 00:23:06,720 --> 00:23:10,160 doors and you know, we start working with a lot of times have 457 00:23:10,160 --> 00:23:14,040 done multiple projects, have had some bloody noses, have had some 458 00:23:14,800 --> 00:23:20,720 hypotheses killed already. So it's also a different dynamic 459 00:23:21,160 --> 00:23:24,880 versus we are thinking about AI. What about you guys? 460 00:23:25,120 --> 00:23:27,760 That's not a fit. No, because they don't know even 461 00:23:27,760 --> 00:23:29,960 what they're looking at. When then we show the platform, 462 00:23:30,160 --> 00:23:33,920 they're like the buttons work and the guys are nice, but we 463 00:23:33,920 --> 00:23:36,400 don't know what to do with this piece of technology, right? 464 00:23:37,000 --> 00:23:39,640 Is that then also kind of the maturity progress an 465 00:23:39,640 --> 00:23:42,320 organization goes through? They see AI, they start 466 00:23:42,320 --> 00:23:45,120 experimenting small, and then they see kind of what it means 467 00:23:45,320 --> 00:23:48,320 with regards to use cases before they put it into production in a 468 00:23:48,320 --> 00:23:49,960 controlled manner. Yeah, and they see like the 469 00:23:50,040 --> 00:23:53,360 projects getting slowed down when going into production or 470 00:23:53,360 --> 00:23:55,960 not making production at all or in production getting some 471 00:23:55,960 --> 00:23:58,080 bloody noses of things going wrong. 472 00:23:58,560 --> 00:24:01,040 And then when they see other platform, they're like, oh, 473 00:24:01,040 --> 00:24:02,920 shit, we needed this six months ago. 474 00:24:03,280 --> 00:24:06,280 But when we would show them the six months before, they would 475 00:24:06,280 --> 00:24:08,400 not recognize the the value of it, right. 476 00:24:08,400 --> 00:24:11,440 So they understand the concepts that we then explain, Oh yeah, 477 00:24:11,440 --> 00:24:13,520 we need the experimentation, we need observability. 478 00:24:13,840 --> 00:24:17,000 But you know, once you've had a couple of those bloody noses, 479 00:24:17,000 --> 00:24:19,400 then you're like, oh, I really need this. 480 00:24:20,320 --> 00:24:21,880 So that's part of the maturity curve. 481 00:24:21,880 --> 00:24:24,360 And we, yeah, then we always are like, hey, let's check back in 482 00:24:24,360 --> 00:24:28,160 six months or let us help you. But yeah, you're you're not 483 00:24:28,160 --> 00:24:29,920 going to get the value out of the platform. 484 00:24:30,400 --> 00:24:32,640 Just are starting and thinking about AII. 485 00:24:33,200 --> 00:24:36,880 Feel like that's like it's it's interesting that organizations 486 00:24:36,880 --> 00:24:41,160 need to experiment, fail and learn in this kind of manner 487 00:24:41,160 --> 00:24:43,960 because it's so accessible and people can try out and people 488 00:24:43,960 --> 00:24:46,760 can kind of see in their own domain what would be valuable 489 00:24:46,760 --> 00:24:48,800 and what would not be valuable and actually start doing that. 490 00:24:49,120 --> 00:24:52,920 It's very different from let's say data-driven solutions. 491 00:24:52,920 --> 00:24:54,840 With machine learning, you needed to have a full 492 00:24:54,840 --> 00:24:57,840 infrastructure layer and histories of data beforehand. 493 00:24:58,200 --> 00:25:01,320 And now you just generate stuff based on what you already have. 494 00:25:01,320 --> 00:25:04,160 Or you tweak your content. If you're you have a marketing 495 00:25:04,160 --> 00:25:06,480 tool and you personalize it based on demographic data that 496 00:25:06,480 --> 00:25:08,360 you have. I feel like people can get up 497 00:25:08,360 --> 00:25:11,360 and running faster and then this cycle of iteration of failing, 498 00:25:11,360 --> 00:25:13,960 getting feedback and improving, they can do that on their own 499 00:25:13,960 --> 00:25:18,400 nowadays. Yeah, because it's a lower a 500 00:25:18,400 --> 00:25:21,640 hurdle of entry, right. Generative because you just use 501 00:25:21,640 --> 00:25:24,120 natural language. Everyone can use natural 502 00:25:24,120 --> 00:25:26,360 language to communicate what most people can, right? 503 00:25:27,240 --> 00:25:30,040 Yeah. You didn't have that in ML, in 504 00:25:30,040 --> 00:25:33,200 the, you know, in traditional ML because there you need like, you 505 00:25:33,200 --> 00:25:36,600 know, researchers and PhDs and data scientists and labeled data 506 00:25:36,800 --> 00:25:40,280 to get a feature out here. It's like, yeah, everyone uses 507 00:25:40,280 --> 00:25:43,920 NACHA GPT. And if you yeah, if you just get 508 00:25:43,920 --> 00:25:46,920 the concepts, a lot of people can already start working with 509 00:25:46,920 --> 00:25:50,480 generative AI, which also brings his own challenges because it's 510 00:25:50,480 --> 00:25:53,280 so simple. People underestimate what you 511 00:25:53,280 --> 00:25:56,920 need for business critical features in production, right? 512 00:25:57,040 --> 00:25:59,960 Someone that says, oh, I use Jet GPT, so we can also put this in 513 00:25:59,960 --> 00:26:01,840 production. No, that's not how it works, 514 00:26:01,840 --> 00:26:05,400 right? Then that's why also requires an 515 00:26:05,400 --> 00:26:09,800 entirely different type of stack than traditional ML needs. 516 00:26:10,080 --> 00:26:12,960 I mean that, I think for me explains why there's so many. 517 00:26:13,200 --> 00:26:16,280 Like the opinions are not very coherent. 518 00:26:16,280 --> 00:26:19,560 It goes all over the place. And I feel like what we see out 519 00:26:19,560 --> 00:26:22,520 there, especially on LinkedIn, but definitely also on Twitter 520 00:26:22,520 --> 00:26:25,680 with regards to different types of thinking and saying, OK, non 521 00:26:25,680 --> 00:26:27,080 engineers can definitely build products. 522 00:26:27,080 --> 00:26:29,560 And like it's, it's so out there and people are either trying to 523 00:26:29,560 --> 00:26:31,640 sell something, they might be bots. 524 00:26:31,840 --> 00:26:34,520 I'm naive as hell and I just see this happening. 525 00:26:34,520 --> 00:26:37,080 I'm like, it's actually I, I don't think it's ever been this 526 00:26:37,080 --> 00:26:40,640 messy with regards to different signals that someone can pick up 527 00:26:40,640 --> 00:26:43,720 one individual and then that helps form their own opinion. 528 00:26:44,360 --> 00:26:47,400 And every cycle you do have these snake oil people, right, 529 00:26:47,400 --> 00:26:51,840 trying to sell snake oil. So I mean, crypto time, right 530 00:26:51,840 --> 00:26:55,720 then when blockchain legit technology, but then you had all 531 00:26:55,720 --> 00:27:00,480 these crypto crap being built or hustled on top of it. 532 00:27:01,600 --> 00:27:04,800 Yeah, this is kind of the new era again, right, Where you have 533 00:27:04,800 --> 00:27:09,280 these people leeching on the people who don't know and just 534 00:27:09,280 --> 00:27:11,200 trying to, yeah, scam them in a way. 535 00:27:11,320 --> 00:27:13,880 Yeah, if someone is. By my e-book. 536 00:27:15,440 --> 00:27:18,600 Yeah, it'll a refund, but you don't get a refund. 537 00:27:19,240 --> 00:27:22,560 If someone is listening and they they hear this and they are 538 00:27:22,560 --> 00:27:25,400 maybe familiar with AI, what would you recommend to them to 539 00:27:25,400 --> 00:27:27,880 actually start getting a fundamental knowledge with 540 00:27:27,880 --> 00:27:30,760 regards to learning what it is, what is out there for their own 541 00:27:30,760 --> 00:27:32,400 organization? Start experimenting. 542 00:27:32,400 --> 00:27:36,720 How would they start? Well, the most important one, of 543 00:27:36,720 --> 00:27:38,920 course, there's so many resources outside. 544 00:27:38,920 --> 00:27:41,280 I mean, you can literally use Chet GPT to get your knowledge 545 00:27:41,280 --> 00:27:47,920 up to speed, right? But to really understand in 546 00:27:47,920 --> 00:27:51,200 which fields this technology can actually help and which ones it 547 00:27:51,200 --> 00:27:55,320 cannot, right? Because whatever question you 548 00:27:55,320 --> 00:27:58,880 ask it, it will answer yes. It doesn't mean yeah, right? 549 00:27:59,160 --> 00:28:02,520 It doesn't mean it's correct. So people think, well, we'll 550 00:28:02,520 --> 00:28:05,840 have AI fix it, right? So yeah, let's plug in AI. 551 00:28:06,560 --> 00:28:09,680 So first of all, it's used as this blanket statement of well, 552 00:28:09,680 --> 00:28:11,520 AI will fix it, which is not the case. 553 00:28:11,840 --> 00:28:14,680 Is really you need to be in the engine room, you know, bloody 554 00:28:14,680 --> 00:28:17,880 noses, dirty hands to really get a feature properly working. 555 00:28:18,760 --> 00:28:20,960 But also understand the boundaries of, OK, you know, 556 00:28:20,960 --> 00:28:25,120 processing natural language, classifying text, all that 557 00:28:25,120 --> 00:28:26,160 stuff. It's great for. 558 00:28:26,160 --> 00:28:29,160 But you know, you cannot just have it run your accounting in 559 00:28:29,160 --> 00:28:32,920 Excel without, you know, with just a prompt. 560 00:28:34,120 --> 00:28:38,280 So really understanding where, how are these models built? 561 00:28:39,080 --> 00:28:41,760 So what are their strengths, but also what are their weaknesses? 562 00:28:42,640 --> 00:28:45,680 So let's double down on the strengths, you know, and map it 563 00:28:45,680 --> 00:28:47,360 to what we need as an organization. 564 00:28:47,760 --> 00:28:51,840 But also let's not fool ourselves by having a use case. 565 00:28:51,840 --> 00:28:54,720 What is actually a weakness of an LLM model, right? 566 00:28:54,880 --> 00:28:57,760 Thinking that again, it's a magic pill that will solve our 567 00:28:58,240 --> 00:29:03,280 accounting processes. You have AI start-ups building 568 00:29:03,280 --> 00:29:06,680 stuff for accounting, but then they're building very custom 569 00:29:06,720 --> 00:29:10,200 solutions for that. But you cannot just, you know, 570 00:29:10,200 --> 00:29:13,840 send an Excel sheet with all your statement bank statements 571 00:29:13,840 --> 00:29:17,960 to a model and then have it, you know, turn out your whole 572 00:29:18,320 --> 00:29:21,120 accounting and and your reports and balance sheets properly. 573 00:29:21,440 --> 00:29:22,240 Gotcha. Yeah. 574 00:29:22,560 --> 00:29:25,160 So the boundaries of the capabilities is important. 575 00:29:25,240 --> 00:29:29,880 Yeah, figuring those, I mean, because it's so accessible, you 576 00:29:29,880 --> 00:29:33,520 can do a lot of experimentation. And I do think with a sceptical 577 00:29:33,520 --> 00:29:36,200 mind you can find those boundaries, even though 578 00:29:36,200 --> 00:29:38,880 everywhere on the Internet sometimes makes it seem that 579 00:29:38,880 --> 00:29:40,760 everything is possible and it's just this magic. 580 00:29:41,320 --> 00:29:43,480 It definitely feels magical, I must say. 581 00:29:44,080 --> 00:29:46,440 I've read a little bit about what goes under the hood, but 582 00:29:46,440 --> 00:29:50,160 from a user perspective, it's like, yeah, it's, it feels 583 00:29:50,240 --> 00:29:53,200 accurate, it feels natural. I've used voice assistants. 584 00:29:53,520 --> 00:29:55,800 They are incredible. Like it just keeps getting 585 00:29:55,800 --> 00:29:58,280 better and better, but critical thinking will always be 586 00:29:58,280 --> 00:30:00,160 necessary. And then you have all these non 587 00:30:00,160 --> 00:30:03,160 functional things as well, right, like rate limits and 588 00:30:04,560 --> 00:30:07,400 token usage and all that stuff. These are all non functionals. 589 00:30:07,400 --> 00:30:11,280 But we have now clients who have very successful features, but 590 00:30:11,280 --> 00:30:13,280 now they're getting crushed by all these rate limits. 591 00:30:13,280 --> 00:30:17,000 Yeah, that currently is right. So that's a limitation of the 592 00:30:17,000 --> 00:30:20,320 technology at this moment. You cannot, you know, service a 593 00:30:20,320 --> 00:30:23,800 million of your users with one API key of one provider. 594 00:30:23,920 --> 00:30:27,040 You will hit rate limits, right? It's the same, you know, I 595 00:30:27,040 --> 00:30:29,840 always say history. Well, I didn't invent the same, 596 00:30:29,840 --> 00:30:32,920 but right, History doesn't repeat, but it rhymes. 597 00:30:34,040 --> 00:30:36,480 During the whole blockchain hype, I was like, yeah, this can 598 00:30:36,480 --> 00:30:39,680 be our centralized database or we're going to, in our company, 599 00:30:39,840 --> 00:30:43,480 use blockchain to store info. Well, and then you had the 600 00:30:43,480 --> 00:30:46,960 problem of the speed of these databases that you cannot use it 601 00:30:46,960 --> 00:30:49,880 as a database for software because it's just too slow, 602 00:30:49,880 --> 00:30:50,800 right. That blockchain. 603 00:30:51,680 --> 00:30:54,560 And why would we, we have a database in our company, right? 604 00:30:54,560 --> 00:30:56,040 Why would we put up a blockchain? 605 00:30:56,040 --> 00:31:00,520 So the the technology being misused for the wrong reasons is 606 00:31:00,680 --> 00:31:05,800 is inherent also allows you to allows us as as mankind to find 607 00:31:05,800 --> 00:31:10,400 new use cases for technology. But you also did not, should not 608 00:31:10,400 --> 00:31:13,680 be afraid to write stuff off because it didn't yield to the 609 00:31:13,680 --> 00:31:16,240 results you wanted. Linking back to the 610 00:31:16,240 --> 00:31:17,960 experimentation mindset you need. 611 00:31:18,080 --> 00:31:20,200 Yeah. One of the operational questions 612 00:31:20,200 --> 00:31:23,480 I had was indeed with regards to rate limits because I've, I've 613 00:31:23,480 --> 00:31:25,400 been in organizations and I've put Gen. 614 00:31:25,400 --> 00:31:28,880 EI solutions to production, but rate limits due to our size, 615 00:31:28,880 --> 00:31:30,960 internal application not reaching a certain level of 616 00:31:30,960 --> 00:31:35,160 skill, not being an app that goes to consumers. 617 00:31:35,160 --> 00:31:38,000 I feel like if you reach a certain skills, rate limits are 618 00:31:38,000 --> 00:31:41,520 definitely going to be an issue. Do you then go and you switch to 619 00:31:41,520 --> 00:31:44,440 a different model or how do you accommodate for rate limits? 620 00:31:44,440 --> 00:31:46,760 Do you really just have to swipe the credit card and pay more? 621 00:31:47,120 --> 00:31:50,360 No. So yes, even that is sometimes 622 00:31:50,360 --> 00:31:55,200 not possible, right? So they're making reserved deals 623 00:31:55,200 --> 00:31:57,360 with the providers, but then you really need to have big 624 00:31:57,360 --> 00:31:59,480 commitments. So as a small startup, you're 625 00:31:59,480 --> 00:32:03,880 not going to like commit to 200 KA year to Google to use to get 626 00:32:03,880 --> 00:32:07,560 dedicated space. Also because you don't know if 627 00:32:07,560 --> 00:32:10,960 you want to be on that model in in two years, maybe a better 628 00:32:10,960 --> 00:32:12,720 one, cheaper one, faster one lock come out. 629 00:32:13,480 --> 00:32:16,360 Yeah, right. So that is one of the reasons. 630 00:32:16,360 --> 00:32:21,000 So what we then do is these fall back orchestration, right? 631 00:32:21,000 --> 00:32:25,600 So in org, for example, people can have yeah, up to five now, 632 00:32:25,600 --> 00:32:28,320 but it can be 10 if you want fall backs. 633 00:32:28,360 --> 00:32:31,640 So if you so you're load balancing your rate limits 634 00:32:31,640 --> 00:32:35,080 across multiple data centers and multiple vendors even, right? 635 00:32:35,080 --> 00:32:39,160 So because a lot of the popular models are provided by multiple 636 00:32:39,160 --> 00:32:41,960 providers, right? So Entropic models, you have 637 00:32:41,960 --> 00:32:45,320 Entropic, but they also run on AWS and on GCP. 638 00:32:45,640 --> 00:32:51,280 So if you put those set up right, like if you use Cloud 4 639 00:32:51,400 --> 00:32:55,920 Opus and you have both, you know, all three providers, you 640 00:32:55,920 --> 00:32:59,640 kind of have an aggregate of the of the rate limits. 641 00:33:00,080 --> 00:33:02,560 It's it's the same model. So you don't have a quality 642 00:33:02,560 --> 00:33:05,040 difference, right. So that's the different ways you 643 00:33:05,040 --> 00:33:08,120 can orchestrate around it. Or you can even decide like as a 644 00:33:08,120 --> 00:33:10,760 fourth fall back, we will electric fall back to open AI 645 00:33:10,760 --> 00:33:14,640 then, which is maybe a different quality or not as good, but at 646 00:33:14,640 --> 00:33:18,080 least we're not down, right. So as a product team, you're 647 00:33:18,080 --> 00:33:20,440 constantly making these trade-offs. 648 00:33:20,960 --> 00:33:23,640 How how good does it need to be? How fast does it need to be? 649 00:33:23,640 --> 00:33:27,920 How cheap does it need to be? And per use case, right? 650 00:33:27,920 --> 00:33:30,240 It's not even per clients, but per use case. 651 00:33:30,440 --> 00:33:33,000 You make different trade-offs. If you're building a customer 652 00:33:33,000 --> 00:33:36,720 facing chat bots, yeah, you, you cannot wait 25 seconds for every 653 00:33:36,720 --> 00:33:39,040 answer. So then speed is of importance. 654 00:33:39,320 --> 00:33:42,280 But if you're running some background process, you don't 655 00:33:42,280 --> 00:33:45,800 care if it's three seconds or 30 seconds, as long as it's proper 656 00:33:45,800 --> 00:33:48,520 processed properly. So then you're optimizing for 657 00:33:48,520 --> 00:33:51,600 correctness, right? And as a product team, you're 658 00:33:51,600 --> 00:33:54,040 always making trade-offs. You do that also in software, 659 00:33:54,040 --> 00:33:55,960 right? Like, do we want it to be fast? 660 00:33:55,960 --> 00:33:58,600 Do we want to be reliable? Do we need to have multiple zone 661 00:33:58,600 --> 00:34:01,000 availabilities? Yeah, you're making all these 662 00:34:01,200 --> 00:34:03,560 trade-offs constantly. Yeah, I like the saying. 663 00:34:03,560 --> 00:34:05,360 There's no good or bad decisions, there's only 664 00:34:05,360 --> 00:34:07,400 trade-offs, Yeah, and that's basically it. 665 00:34:07,840 --> 00:34:11,159 But I love this solution of there's multiple cloud providers 666 00:34:11,159 --> 00:34:15,080 hosting these same models and then having kind of availability 667 00:34:15,080 --> 00:34:17,440 across that. Yeah, to kind of up your rate 668 00:34:17,440 --> 00:34:19,199 limit. I feel like it's genius to be. 669 00:34:19,199 --> 00:34:20,199 Honest. Yeah, well, for example with 670 00:34:20,199 --> 00:34:23,639 Azure then, right, You, you can literally deploy a model in 671 00:34:23,639 --> 00:34:24,800 different regions. Yeah. 672 00:34:24,800 --> 00:34:27,159 So you actually have the regions as each other's fall backs. 673 00:34:27,360 --> 00:34:30,159 So you have now all of a sudden like your whole European rate 674 00:34:30,159 --> 00:34:35,360 limit of all the different Azure data centers combined? 675 00:34:35,480 --> 00:34:39,040 Yeah, that's really smart. I was only thinking along one 676 00:34:40,080 --> 00:34:42,360 kind of axis, which would be switching models. 677 00:34:42,600 --> 00:34:46,760 But switching models, I think for content, and this is, I mean 678 00:34:46,760 --> 00:34:48,960 conversion optimization always goes to the nitty gritty. 679 00:34:48,960 --> 00:34:52,440 But for content, if you have a marketing tool and that does 680 00:34:52,440 --> 00:34:55,400 personalized content based on user demographics, one model 681 00:34:55,400 --> 00:34:58,880 might outperform another one, but it should be negligible 682 00:34:58,880 --> 00:35:00,800 because we're talking about content here specifically. 683 00:35:00,800 --> 00:35:03,600 So switching a model then is not going to be the end of the 684 00:35:03,600 --> 00:35:06,480 world. But I like that you have the 685 00:35:06,480 --> 00:35:09,080 option to keep the same models consistent and then still 686 00:35:09,080 --> 00:35:10,960 manoeuvre around those rate limits that you have. 687 00:35:11,120 --> 00:35:12,680 Yeah. And then content, I agree. 688 00:35:12,680 --> 00:35:15,280 And even there, right, some people like Claude's creative 689 00:35:15,280 --> 00:35:18,360 writing versus some other language differences, right. 690 00:35:18,360 --> 00:35:21,200 So because we always think in English. 691 00:35:21,200 --> 00:35:24,480 But yeah, if you have a friend we have literally with our 692 00:35:24,480 --> 00:35:28,520 contextual router, you can do contextual prompting. 693 00:35:29,040 --> 00:35:31,880 You can literally say, hey, the French users will send to 694 00:35:31,880 --> 00:35:35,680 Mistral, the English will send to GPT, and the Dutch will send 695 00:35:35,680 --> 00:35:41,080 to Anthropic because the Dutch localization is better in that 696 00:35:41,080 --> 00:35:43,040 language. And Indian we send to an Indian 697 00:35:43,040 --> 00:35:45,080 LLM. Well, Indian is not a language, 698 00:35:45,080 --> 00:35:50,520 but Indian languages will send to this specific LLM for for 699 00:35:50,520 --> 00:35:52,120 localization. Yeah, right. 700 00:35:53,160 --> 00:35:54,600 You all of a sudden need that. Yeah. 701 00:35:55,640 --> 00:35:57,880 I like, I like that a lot to be honest. 702 00:35:57,880 --> 00:36:01,640 Like for me, I, I thought, OK, getting stuff in production, 703 00:36:01,640 --> 00:36:03,920 that's one thing. Having kind of the, the 704 00:36:03,920 --> 00:36:07,200 operations mindset with regards to observability guardrails, 705 00:36:07,200 --> 00:36:09,520 seeing where things kind of go out of the boundaries that we 706 00:36:09,520 --> 00:36:13,200 think we have is one thing. But I've never thought of this. 707 00:36:13,400 --> 00:36:15,440 I mean I have thought of this but I I never thought you could 708 00:36:15,440 --> 00:36:18,240 go that deep with regards to using and leveraging different 709 00:36:18,240 --> 00:36:20,800 models in the same use case, right? 710 00:36:21,000 --> 00:36:23,280 It makes sense to have one use case and then one model will 711 00:36:23,280 --> 00:36:25,960 pick the best model. But your use case might vary so 712 00:36:25,960 --> 00:36:28,800 much that you can have multi model solutions based on locale 713 00:36:29,120 --> 00:36:31,400 like you mentioned with regards to different writing styles. 714 00:36:31,400 --> 00:36:34,480 Even data residency, right? So we have clients that like 715 00:36:35,200 --> 00:36:38,680 different business units, the data is not allowed to leave 716 00:36:38,920 --> 00:36:42,600 their region. So they will literally route use 717 00:36:42,600 --> 00:36:47,160 cases to an LLM deployed in a specific region, right? 718 00:36:47,760 --> 00:36:50,600 And they need that orchestration all of a sudden so it can go 719 00:36:50,840 --> 00:36:53,400 really crazy. That's because data residency in 720 00:36:53,400 --> 00:36:57,560 AI becomes more and more of a criteria in every decision. 721 00:36:57,800 --> 00:36:59,200 It's definitely a hot topic. Right. 722 00:36:59,200 --> 00:37:01,080 Yeah, yeah, right. Where does the model run? 723 00:37:01,360 --> 00:37:04,640 Is literally a thing. And then we now have Australian 724 00:37:04,640 --> 00:37:07,200 healthcare companies. It cannot literally leave the 725 00:37:07,200 --> 00:37:10,600 boundaries of Australia, right? And India has the same type of 726 00:37:10,600 --> 00:37:13,560 laws and regulations. Well, GDPR, we all know, right? 727 00:37:13,560 --> 00:37:17,200 It's all like, like physically it's not allowed to leave. 728 00:37:17,200 --> 00:37:20,360 So how do you guarantee that and make sure that people have that 729 00:37:20,360 --> 00:37:22,600 granular control? And of course, if you're just 730 00:37:22,600 --> 00:37:25,800 summarizing some SEO marketing text, you don't really care 731 00:37:25,800 --> 00:37:27,280 whether you send it to the US or not. 732 00:37:27,280 --> 00:37:30,240 But if you like summarizing healthcare records, which 733 00:37:30,240 --> 00:37:34,000 clients of US do in healthcare, yeah, it literally physically is 734 00:37:34,000 --> 00:37:36,040 not allowed to leave the boundaries of the EU, right? 735 00:37:36,160 --> 00:37:38,200 Yeah. So you get those type of a non 736 00:37:38,200 --> 00:37:39,520 functional requirements. Yeah. 737 00:37:40,160 --> 00:37:42,960 As a last thought, I wanted to ask you this question because I 738 00:37:42,960 --> 00:37:46,200 see a lot of companies that are starting up and trying to do 739 00:37:46,200 --> 00:37:48,400 something with regards to Gen. AI, right, Trying to find 740 00:37:48,400 --> 00:37:51,680 product market fit, either B to BB to C, anything they can find. 741 00:37:52,120 --> 00:37:54,560 What would be your advice with regards to the challenges 742 00:37:54,560 --> 00:37:57,200 they're going to face or what to look out for when they're trying 743 00:37:57,200 --> 00:38:03,920 to go on that path? Find the right people and what I 744 00:38:03,920 --> 00:38:07,000 mean by that is try to stay away from all these snake oil sales 745 00:38:07,000 --> 00:38:09,360 people, right? Because they will apply to jobs, 746 00:38:09,600 --> 00:38:15,080 especially if you put up an AI job ad now the the top, top tier 747 00:38:15,080 --> 00:38:18,680 people will not apply. But then you attract all these. 748 00:38:19,600 --> 00:38:23,000 Well, not everyone applying, but you know, it's a higher risk 749 00:38:23,000 --> 00:38:28,280 that you attract also some wannabes and because companies 750 00:38:28,280 --> 00:38:31,960 are so desperate, especially if you have difficulty attracting 751 00:38:31,960 --> 00:38:35,120 the right talent, you're not a tech first, AI first company, 752 00:38:36,600 --> 00:38:39,320 then you're like, OK, I hope this version is the magic pill, 753 00:38:39,320 --> 00:38:40,640 right? Then they sell themselves really 754 00:38:40,640 --> 00:38:42,120 well. And then you would think, oh, 755 00:38:42,120 --> 00:38:45,040 they're going to help us with our AI strategy and it won't. 756 00:38:45,240 --> 00:38:48,520 You as a leader also need to be in the trenches and get your 757 00:38:48,520 --> 00:38:50,840 hands dirty. So attracting the right people 758 00:38:51,120 --> 00:38:55,880 and then just starting, right, just start developing hypothesis 759 00:38:55,880 --> 00:39:01,960 and testing and set yourself up for that continuous testing as 760 00:39:01,960 --> 00:39:03,800 well. But that comes maybe in Step 2. 761 00:39:03,800 --> 00:39:06,760 Step 1 is just get going with the right people. 762 00:39:06,840 --> 00:39:08,880 Gotcha. Now how, how do you do that? 763 00:39:08,880 --> 00:39:10,600 Do you find all the people through your network? 764 00:39:10,600 --> 00:39:13,040 You already mentioned juniors. In the phase that you're in is 765 00:39:13,040 --> 00:39:14,800 not really what you're looking for. 766 00:39:14,800 --> 00:39:17,200 Yeah, we have. I can remember last time we put 767 00:39:17,200 --> 00:39:22,320 out a job ad, it's always so our best top tier performers we have 768 00:39:22,320 --> 00:39:25,960 in the company, they just walked into the office like, guys, I 769 00:39:25,960 --> 00:39:27,520 want to work here, I don't care what. 770 00:39:28,160 --> 00:39:31,480 And as far as they also had yeah, yeah, yeah, or yeah, or 771 00:39:31,480 --> 00:39:33,320 through network, they make sure they get an intro. 772 00:39:33,640 --> 00:39:37,040 So it's always been people applying and not even like, hey, 773 00:39:37,040 --> 00:39:40,280 here's my resume, but more like let's talk. 774 00:39:40,360 --> 00:39:43,160 And then it matches. Also because we're not 775 00:39:43,160 --> 00:39:46,800 recruiting 10s of hundreds of people, so we only need a couple 776 00:39:46,800 --> 00:39:52,600 really good ones and we're in a sexy space at the moment. 777 00:39:53,160 --> 00:39:55,440 So yeah, we people come to us, Yeah. 778 00:39:56,040 --> 00:39:58,280 In that front, yeah. Even though you don't have a job 779 00:39:58,280 --> 00:40:00,360 position open, would people still come and you would still 780 00:40:00,360 --> 00:40:03,320 hire those people? Right, talent, we make it work 781 00:40:03,720 --> 00:40:06,480 also because we're always looking for, you know, what we 782 00:40:06,480 --> 00:40:09,640 call M profiles, people that have broad interests and 783 00:40:09,640 --> 00:40:12,760 capabilities and actually can go deep on a couple of fronts. 784 00:40:12,760 --> 00:40:13,400 Gotcha. Right. 785 00:40:13,400 --> 00:40:19,040 On AI or on engineering, but they're also just mini CEOs on 786 00:40:19,040 --> 00:40:20,160 their own. Yeah, right. 787 00:40:20,400 --> 00:40:23,480 Most of our people are from day one have been saying one day I'm 788 00:40:23,480 --> 00:40:26,120 going to have my own company. I'm here to learn and I'm 789 00:40:26,120 --> 00:40:28,760 totally supporting that. I would love to, you know, be 790 00:40:28,760 --> 00:40:33,440 their Angel investor one day. So yeah, that's the type we're 791 00:40:34,000 --> 00:40:38,800 attracting and working with. It's all these mini future 792 00:40:38,800 --> 00:40:40,800 founders themselves. That's awesome. 793 00:40:41,200 --> 00:40:42,720 Thank you so much for coming on and sharing. 794 00:40:42,720 --> 00:40:45,600 So this was amazing. I I love that it was a lot of 795 00:40:45,600 --> 00:40:46,800 fun. I must say it was good. 796 00:40:47,400 --> 00:40:49,120 All right then we're going to round it off here. 797 00:40:49,120 --> 00:40:51,440 Thank you so much. The best way to support the show 798 00:40:51,440 --> 00:40:53,520 is to leave a like. Whether you're on Spotify, apple 799 00:40:53,520 --> 00:40:55,320 pie cast or on YouTube doesn't really matter. 800 00:40:55,520 --> 00:40:57,680 Leave a like anywhere and we'll see you on the next one.