1 00:00:08,360 --> 00:00:11,480 Hello and welcome you all. Today I'm having Matas Rastiis 2 00:00:11,480 --> 00:00:13,800 with me. Matas is Staff Software Engineer 3 00:00:13,800 --> 00:00:17,360 at Uber's developer platform, and he's going to share how Uber 4 00:00:17,360 --> 00:00:20,240 evaluates, deploys, and measures the best AI tools to make 5 00:00:20,240 --> 00:00:22,560 developers as productive as possible. 6 00:00:22,840 --> 00:00:25,680 Let's dig into it. Oh, and of course you can watch 7 00:00:25,680 --> 00:00:28,760 the whole conversation on Spotify, YouTube and now even on 8 00:00:28,760 --> 00:00:32,560 Apple Podcasts. Also, Mata's job description 9 00:00:32,560 --> 00:00:35,680 change and he got a very cool promotion, so give him a shout 10 00:00:35,680 --> 00:00:37,680 out on LinkedIn. Let's go. 11 00:00:38,640 --> 00:00:41,480 Welcome to the podcast Mata's. Thank you so much for having me. 12 00:00:41,480 --> 00:00:44,120 Thank you. I wanted to make an episode 13 00:00:44,120 --> 00:00:49,600 about like the topic that I guess most companies are on to 14 00:00:49,600 --> 00:00:52,680 right now, which is like how to augment the workforce, like how 15 00:00:52,680 --> 00:00:56,000 to really get most of the AI tools and how to really 16 00:00:56,000 --> 00:00:59,640 distribute them onto the workforce, onto the engineering 17 00:00:59,640 --> 00:01:03,800 people to make most out of it. So I'm really glad to have you 18 00:01:03,800 --> 00:01:05,800 here and talk about your work at Uber. 19 00:01:06,240 --> 00:01:09,320 Awesome. Thanks for having me shoot away. 20 00:01:10,040 --> 00:01:12,880 Before we dig into that, maybe we could have a quick 21 00:01:12,880 --> 00:01:14,560 introduction to who you are and what. 22 00:01:14,560 --> 00:01:17,160 Your job is sounds good. My name is Matos Restanos. 23 00:01:17,520 --> 00:01:22,160 I work at Uber and we're specifically in the developer 24 00:01:22,160 --> 00:01:25,720 platform organization of Uber. So we help engineers be more 25 00:01:25,720 --> 00:01:30,000 productive in any way we can. Nowadays that's that means 26 00:01:30,000 --> 00:01:34,480 enabling AI, you know, as as much and as fast as we can, 27 00:01:36,160 --> 00:01:40,520 scaling engineers leverage to to new heights to where they can 28 00:01:40,520 --> 00:01:44,040 now author, you know, two 3 * X more code than they could 29 00:01:44,040 --> 00:01:46,320 before. Yeah. 30 00:01:46,680 --> 00:01:51,480 And I work on a variety of different things that include, 31 00:01:51,480 --> 00:01:55,280 you know, coding assistance, automatic test generation 32 00:01:55,280 --> 00:01:59,600 tooling, AR guard rails and background agents as well. 33 00:02:00,280 --> 00:02:03,120 So things you can launch and sort of leave or chat with on 34 00:02:03,120 --> 00:02:05,960 your phone so you don't have to have your laptop even open to 35 00:02:05,960 --> 00:02:07,960 code. That's the overview. 36 00:02:09,080 --> 00:02:11,640 Interesting. Like there are so many new 37 00:02:11,640 --> 00:02:16,600 technologies, so many new tools, so many new AI things coming out 38 00:02:16,600 --> 00:02:18,480 at the moment. Like if there's something new 39 00:02:18,480 --> 00:02:21,880 coming out next week, what would the workflow look like when it 40 00:02:21,880 --> 00:02:24,320 comes to giving it to the people? 41 00:02:24,320 --> 00:02:26,480 Like really making the most out of it? 42 00:02:26,480 --> 00:02:29,880 Like do you have a top down approach or just engineers free 43 00:02:29,880 --> 00:02:33,320 to choose whatever they want and you have a bottom up? 44 00:02:33,320 --> 00:02:35,000 Like how does it? Look like, yeah, yeah. 45 00:02:36,160 --> 00:02:41,440 So we found success in gauging sort of the popular interest and 46 00:02:41,440 --> 00:02:45,400 then the vibes, if you could say I could, I could call them that 47 00:02:46,360 --> 00:02:49,920 out on the interwebs. So you can imagine, you know, X 48 00:02:49,920 --> 00:02:54,360 or Twitter sentiment, other companies adopting tools. 49 00:02:54,600 --> 00:02:56,360 We tried to stay at the very forefront. 50 00:02:56,360 --> 00:02:59,480 So if you have a big wave coming, coming on, there's 51 00:02:59,480 --> 00:03:02,480 usually, you know, Hacker News articles, We've got a really 52 00:03:02,680 --> 00:03:07,600 sort of plugged in team that's essentially forwards like new 53 00:03:07,960 --> 00:03:11,440 like interesting lab leading lab posts immediately. 54 00:03:11,440 --> 00:03:14,080 So you get like within an hour, you know, whether you know, Opus 55 00:03:14,080 --> 00:03:18,760 4.6 drops or something, or if there's someone like Openclaw 56 00:03:18,760 --> 00:03:21,120 releases. You also get like in the theme 57 00:03:21,120 --> 00:03:23,160 channel itself, you get the the piece of news. 58 00:03:23,160 --> 00:03:25,360 So you can read them. I read the news and like, oh, 59 00:03:25,480 --> 00:03:27,120 you know, we need to get on this. 60 00:03:27,120 --> 00:03:28,840 Or maybe this is, you know, overhead. 61 00:03:28,840 --> 00:03:31,840 Do you have specific criteria on how to evaluate whether you want 62 00:03:31,840 --> 00:03:33,400 to dig into a certain topic or not? 63 00:03:33,560 --> 00:03:35,120 Yeah. That's a really good question. 64 00:03:35,680 --> 00:03:40,240 We actually have, we sort of have this funnel, it's not fully 65 00:03:40,240 --> 00:03:43,160 built, built out yet, but the idea is at the very top of the 66 00:03:43,160 --> 00:03:44,560 funnel. You have sort of popular 67 00:03:44,560 --> 00:03:48,040 sentiment and then you can look at, for example, benchmark 68 00:03:48,040 --> 00:03:49,760 websites. So we have, you know, suite 69 00:03:49,760 --> 00:03:51,760 bench. There's another one that's I 70 00:03:51,760 --> 00:03:56,160 forget the name on of that do daily and weekly and monthly 71 00:03:56,160 --> 00:04:01,360 tracking of different, you know, a genetic CL is X different 72 00:04:01,360 --> 00:04:03,680 models that those CL is can run on. 73 00:04:04,160 --> 00:04:07,560 And if you see, for example, a new offering, I don't know from 74 00:04:07,560 --> 00:04:10,640 China, like deep sea popping up, you might want to validate 75 00:04:10,640 --> 00:04:13,640 through public benchmarks. So they run a bunch of like bug 76 00:04:13,640 --> 00:04:17,440 tasks against say the, the Linux repository or Chrome or 77 00:04:17,440 --> 00:04:21,519 something. And then you get a good picture 78 00:04:22,079 --> 00:04:26,080 and a good initial filter. So you can stop, you know, like 79 00:04:26,120 --> 00:04:28,640 if if people say it kind of sucks, you cut it off there. 80 00:04:29,080 --> 00:04:32,840 And then the bottom of the funnel gets into internal 81 00:04:32,840 --> 00:04:39,720 benchmarking, which is you take, you know, sort of larger or, or 82 00:04:39,960 --> 00:04:41,640 bloopers repos are large in scale. 83 00:04:41,920 --> 00:04:45,840 So you take tasks from those repos specifically and they 84 00:04:45,840 --> 00:04:48,080 also, you know, it, it's not an open source repository. 85 00:04:48,080 --> 00:04:50,080 So we can't just like clone the repo and build it. 86 00:04:50,400 --> 00:04:52,880 And so as an example, you know, you have a lot of companies 87 00:04:53,200 --> 00:04:57,120 offering background agents now, but all of them depend on the 88 00:04:57,280 --> 00:05:00,000 companies providing compute by themselves to the background 89 00:05:00,000 --> 00:05:02,560 agent. So what happens then is we can't 90 00:05:02,560 --> 00:05:05,560 actually leverage it because our repos are not hermetic. 91 00:05:05,560 --> 00:05:07,720 They can't be built outside of Ubers and Fed that easily. 92 00:05:08,560 --> 00:05:11,160 They can't test, can't run on a bunch of other problems. 93 00:05:11,560 --> 00:05:17,360 So that just crosses out that whole class of agents, even 94 00:05:17,360 --> 00:05:20,440 though on the open source side of things it they seems to be 95 00:05:20,440 --> 00:05:22,440 working fine, you know, completing tasks and whatnot. 96 00:05:22,760 --> 00:05:25,400 So that's the the sort of middle layer it's automated 97 00:05:25,400 --> 00:05:29,760 benchmarking for on the internal side of things. 98 00:05:30,200 --> 00:05:34,400 And now the third level would be just user like focus groups, 99 00:05:34,400 --> 00:05:39,080 giving, you know, a set of power users, the tool, seeing what 100 00:05:39,080 --> 00:05:40,880 they think. If the feedback is negative, 101 00:05:40,880 --> 00:05:42,960 it's also likely that no one else will care. 102 00:05:42,960 --> 00:05:45,280 Like, why do we even bother investing in this? 103 00:05:46,600 --> 00:05:50,000 If the feedback is positive or very positive in the case of 104 00:05:50,000 --> 00:05:53,520 some tools that we had recently, you get a, you know, you just 105 00:05:53,520 --> 00:05:55,880 start pushing it. So like make it available to 106 00:05:55,880 --> 00:05:59,440 people. And then perhaps if if the 107 00:05:59,680 --> 00:06:03,000 adoption curve is is not steep enough, you can do workshops and 108 00:06:03,560 --> 00:06:06,280 hack days and whatnot. Let's say people are really 109 00:06:06,280 --> 00:06:09,560 actually using it. Do you have sort of KP is like 110 00:06:09,560 --> 00:06:13,880 measuring developer productivity or the outcome of using those 111 00:06:13,880 --> 00:06:16,280 tools and what what are these? Yeah, yeah. 112 00:06:16,480 --> 00:06:22,200 So the classic OK R or sort of metric that is predominant or 113 00:06:22,200 --> 00:06:25,280 used to be predominant is essentially diff count or code 114 00:06:25,280 --> 00:06:28,200 change count. So or, you know, get that VR 115 00:06:28,200 --> 00:06:32,640 count and there's various symmetrics, like how many of 116 00:06:32,640 --> 00:06:37,040 your code changes are X size so that you don't have like a bunch 117 00:06:37,040 --> 00:06:38,640 of small little nitpicky changes. 118 00:06:39,040 --> 00:06:45,720 But to, to zoom back out, if you look at a group of engineers and 119 00:06:46,080 --> 00:06:49,360 some of them are adopting a certain tool and that metric 120 00:06:49,360 --> 00:06:52,440 goes up, you can say at least that they're producing more code 121 00:06:52,440 --> 00:06:54,480 changes of, you know, certain size. 122 00:06:55,680 --> 00:06:57,600 That's a good proxy metric for productivity. 123 00:06:57,640 --> 00:07:01,840 It's not actually it, it's not connected to the end goal of the 124 00:07:01,840 --> 00:07:05,720 company, which is shipping more features, improving the app, but 125 00:07:05,720 --> 00:07:08,920 it's a proxy. And we've been trying to link, 126 00:07:09,720 --> 00:07:14,000 there's some interesting work going on to actually link those 127 00:07:14,040 --> 00:07:17,920 that that proxy metric to a more concrete like, hey, you know, 128 00:07:18,200 --> 00:07:20,800 these features were shipped because of these code changes. 129 00:07:20,960 --> 00:07:24,560 So then you can look at feature velocity and like essentially 130 00:07:24,560 --> 00:07:29,000 map more towards like a actual gross booking value outcome as 131 00:07:29,000 --> 00:07:33,040 opposed to just looking at, you know, discount and assuming that 132 00:07:33,040 --> 00:07:35,600 it, it will result in better outcomes for the company. 133 00:07:35,960 --> 00:07:39,040 So that's one. And then there's other angles 134 00:07:39,040 --> 00:07:44,640 you can look at for just optimizing the engineer's time 135 00:07:44,640 --> 00:07:46,960 spent developing and doing other tasks. 136 00:07:47,960 --> 00:07:51,840 So things like tracking how much they spend on docs, you know, 137 00:07:51,920 --> 00:07:54,400 are they like reading the meeting notes, for example. 138 00:07:54,840 --> 00:07:58,040 This is goes a bit outside of coding as well, but do they 139 00:07:58,040 --> 00:08:00,480 spend a lot of time in ticketing systems creating tickets? 140 00:08:00,480 --> 00:08:01,760 Do we need to help them with that? 141 00:08:02,600 --> 00:08:05,840 This is sort of really early that tracking is a complicated 142 00:08:05,840 --> 00:08:10,600 system to set up, but we hope that it will enable us to even 143 00:08:10,600 --> 00:08:13,280 spot bottlenecks that we didn't know existed before. 144 00:08:13,680 --> 00:08:16,840 Like why is someone spending so much on, you know, this Google 145 00:08:16,840 --> 00:08:20,200 doc or like AG ratio? I can imagine, you know, JIRA 146 00:08:20,200 --> 00:08:23,880 being like a pinpoint for some people, but I, I know that 147 00:08:23,880 --> 00:08:26,360 there's things that we don't even know that are bottlenecks 148 00:08:26,360 --> 00:08:27,520 that they're trying, We're trying to. 149 00:08:27,520 --> 00:08:31,880 Discover, can you walk us through like what maybe right 150 00:08:31,880 --> 00:08:38,400 now the the workflow is for an engineer like how did you what 151 00:08:38,400 --> 00:08:40,640 type of augmentation have you already implemented in the 152 00:08:40,640 --> 00:08:43,480 developer experience when it comes to delivering features? 153 00:08:43,720 --> 00:08:47,120 Yeah, so we have a lot of different tools. 154 00:08:47,120 --> 00:08:50,360 I don't have a way to flash like a graphic of everything we have, 155 00:08:50,680 --> 00:08:53,640 but you know, from the very beginning, like planning and 156 00:08:53,640 --> 00:08:56,320 designing stages, you've got your, you know, E augmented 157 00:08:56,720 --> 00:08:58,640 prototyping tools. We've got Figma but. 158 00:08:58,640 --> 00:09:01,400 If you have a graphic, we could just put it into the episode 159 00:09:01,400 --> 00:09:03,360 later on. That's what I'm always saying. 160 00:09:03,360 --> 00:09:05,600 At least we're in the end. I'm already forgetting to put 161 00:09:05,600 --> 00:09:06,600 it. Yeah, yeah, yeah. 162 00:09:06,920 --> 00:09:08,560 Gotcha. I'll see if I can pull something 163 00:09:08,560 --> 00:09:10,840 up. But anyway, for designing you've 164 00:09:10,840 --> 00:09:12,400 got your figma, your lovable, whatever. 165 00:09:12,640 --> 00:09:17,480 We we're have this new explosion of POC capability now to where 166 00:09:17,920 --> 00:09:21,160 you can bring a whole quote UN quote working product or like a 167 00:09:21,200 --> 00:09:24,760 demo of a product into a meeting as opposed to like a big idea or 168 00:09:24,760 --> 00:09:29,720 like a SketchUp in Excalibur. So that's a very fun new 169 00:09:29,720 --> 00:09:32,480 capability that's unlocked because the cost of them is so 170 00:09:32,480 --> 00:09:34,680 low now with agenda coding assistance. 171 00:09:35,080 --> 00:09:39,880 But yeah, so past the design stage, you've got, you've got AI 172 00:09:39,880 --> 00:09:42,000 enabled ticketing systems now. So you've got linear, for 173 00:09:42,000 --> 00:09:45,160 example, you can do instead of going through the web UI like 174 00:09:45,160 --> 00:09:47,920 messing around with creating a ticket, you can do, you know, in 175 00:09:47,920 --> 00:09:51,120 Slack at a linear, make a ticket for this item. 176 00:09:51,880 --> 00:09:54,760 That's pretty cool. And then past the design stages, 177 00:09:54,760 --> 00:09:58,840 when you get into actual coding, we have it, it depends on user 178 00:09:58,840 --> 00:10:01,720 cohort. So the most we we have, you 179 00:10:01,720 --> 00:10:05,680 know, power users and sort of more regular engineers or, or or 180 00:10:05,680 --> 00:10:08,320 regular AI adapters for power users. 181 00:10:08,320 --> 00:10:13,760 The predominant sort of pathway to develop now is agentics. 182 00:10:13,760 --> 00:10:18,400 CL is so that it splinters into a few different offerings. 183 00:10:18,400 --> 00:10:25,040 You've got clock code, you've got codecs, CLI and for the real 184 00:10:25,040 --> 00:10:27,920 like massively parallel work streams. 185 00:10:28,920 --> 00:10:31,600 The power users are able to paralyze those agents through 186 00:10:31,600 --> 00:10:35,360 something like AT Max terminal manager to where you have, you 187 00:10:35,360 --> 00:10:38,240 know, let's say like six of those agents working for you at 188 00:10:38,240 --> 00:10:40,160 the same time. They've got notification set up. 189 00:10:40,280 --> 00:10:44,760 So they notify you, the engineer when they're done so you can go 190 00:10:44,760 --> 00:10:47,320 and switch back and continue work. 191 00:10:49,280 --> 00:10:53,240 And yeah, and we are trying to enable engineers to just tell 192 00:10:53,240 --> 00:10:55,120 their agents what to do. So like, hey, publish my change. 193 00:10:55,400 --> 00:10:57,280 And then the agents themselves need to have the know how. 194 00:10:57,280 --> 00:11:01,200 So whether it's be whether it be skills, throntropics, products 195 00:11:01,720 --> 00:11:06,040 or commands, you know, cursor and whatnot to let's say create 196 00:11:06,040 --> 00:11:10,400 the PR or the DIF so that the engineer doesn't have to spend 197 00:11:10,600 --> 00:11:14,560 more time like making a description, calling the Clis to 198 00:11:14,560 --> 00:11:16,040 do it. It's just done for them. 199 00:11:16,240 --> 00:11:18,160 And if they queue up enough actions like hey, you know, make 200 00:11:18,160 --> 00:11:21,400 the change, review it, produce a report of how you validated it, 201 00:11:21,400 --> 00:11:24,320 maybe do the like an end to end manual test and then publish the 202 00:11:24,320 --> 00:11:26,880 change. You can queue up, let's say an 203 00:11:26,880 --> 00:11:30,240 hour and a half worth of actions to where now that whole session 204 00:11:30,240 --> 00:11:33,880 is parallel and just running. It's like a teammate that's just 205 00:11:33,920 --> 00:11:35,840 doing its work. If it gets stuck, it alerts you 206 00:11:36,360 --> 00:11:39,880 and you could check on it and you know, it gets to essentially 207 00:11:39,880 --> 00:11:42,800 an end state. So that's power users and 208 00:11:44,160 --> 00:11:46,840 general AI adoption. I think so far we've seen cursor 209 00:11:46,840 --> 00:11:50,120 and sort of a genetic IDs be a primary interface for those. 210 00:11:50,480 --> 00:11:54,640 So and you can imagine VS Code plus any of your extensions 211 00:11:54,640 --> 00:11:57,160 compiled or cloud code as an extension and VS Code is nice 212 00:11:58,080 --> 00:12:02,480 cursor of course, a product as well that's AI enabled. 213 00:12:03,440 --> 00:12:05,880 So you do your coding there. It's primarily single threaded 214 00:12:05,880 --> 00:12:07,960 still. So you have your code view, you 215 00:12:07,960 --> 00:12:11,320 can do your editing yourself and then you have a chat session. 216 00:12:11,400 --> 00:12:13,440 You can have multiple chat sessions across multiple files 217 00:12:13,440 --> 00:12:17,680 as well as well. But it's yeah. 218 00:12:17,840 --> 00:12:20,080 And like orchestrating the multiple repository 219 00:12:20,080 --> 00:12:24,000 implementation details is also a topic that more of the power 220 00:12:24,000 --> 00:12:26,880 user cohort gets into and gets really resolved so that you can 221 00:12:26,880 --> 00:12:29,240 do multiple issues on the same repository at the same time. 222 00:12:30,000 --> 00:12:34,680 Meanwhile, the regular users kind of stick to a single repo 223 00:12:34,680 --> 00:12:38,120 or maybe a couple clones. Yeah. 224 00:12:38,120 --> 00:12:43,520 And then past that, we have at diff review or PR review time, 225 00:12:43,640 --> 00:12:46,680 you have tooling to go over your diff and spot bugs. 226 00:12:46,960 --> 00:12:49,240 So you we have our internal tool called you review. 227 00:12:49,920 --> 00:12:55,040 We have cursor bug bot running, so spotting things that you 228 00:12:55,040 --> 00:12:57,520 might have missed in your, you know, publishing steps. 229 00:12:58,560 --> 00:13:01,440 And then that's it. Then you go to human review and 230 00:13:01,440 --> 00:13:04,360 you line the code change. And then there's really quickly 231 00:13:04,680 --> 00:13:09,640 final sort of side tangent that it's going to grow a lot in 232 00:13:09,640 --> 00:13:12,080 adoption in the coming year or two. 233 00:13:12,560 --> 00:13:15,320 And we're actually expected to become the primary means of 234 00:13:15,320 --> 00:13:17,520 editing code and that is the background agents. 235 00:13:17,840 --> 00:13:20,200 So you, instead of even creating your own development 236 00:13:20,200 --> 00:13:23,200 environment, unless you're like the superpower user that knows, 237 00:13:23,240 --> 00:13:25,280 essentially orchestrates the background agents already in 238 00:13:25,280 --> 00:13:27,760 your background in your environment, like dev box. 239 00:13:28,200 --> 00:13:32,520 You, you have AUI or you start to slack or something, you know, 240 00:13:32,520 --> 00:13:34,360 add an agent and you start a task. 241 00:13:34,440 --> 00:13:37,880 So like give it an issue or something and then it doesn't 242 00:13:37,880 --> 00:13:39,680 work in the background for you. And then 40 minutes later you 243 00:13:39,680 --> 00:13:41,120 get a notification for, Hey, I'm done. 244 00:13:41,600 --> 00:13:45,160 And you can like zoom in, edit the code, follow up, give it 245 00:13:45,160 --> 00:13:48,800 another hour of work to to go. And then if we, if you have a 246 00:13:48,800 --> 00:13:50,920 nice interface for those, you can have like again, six to ten 247 00:13:50,920 --> 00:13:52,240 of those agents running in parallel. 248 00:13:52,600 --> 00:13:56,720 It really gets into more of the human, the limit of how many 249 00:13:56,720 --> 00:13:58,840 parallel things you can think about at the same time because 250 00:13:58,840 --> 00:14:00,720 you need to have this rolodex of ideas. 251 00:14:01,240 --> 00:14:03,600 Now I'm working on this thing and now I'm working on something 252 00:14:03,600 --> 00:14:08,120 else and just be able to do that to support this workflow. 253 00:14:08,320 --> 00:14:12,160 But we're hoping that engineers will train that muscle because 254 00:14:12,160 --> 00:14:16,880 to maximally lever up like we have this, the nascent concept 255 00:14:16,880 --> 00:14:19,280 of like the parallel parallelization factor of an 256 00:14:19,280 --> 00:14:21,480 engineer, how, how many work streams can you do at the same 257 00:14:21,480 --> 00:14:23,280 time? So instead of just doing 1 258 00:14:23,480 --> 00:14:26,240 synchronously like you used to do before, maybe it's not 2 or 259 00:14:26,320 --> 00:14:28,560 perhaps probably it's three like a median. 260 00:14:28,960 --> 00:14:31,000 I know some people can go up to 67810, whatever. 261 00:14:31,280 --> 00:14:34,080 But it's, that's, it's really hard to even keep the agents 262 00:14:34,080 --> 00:14:36,080 busy for that long because you're going to have to review 263 00:14:36,080 --> 00:14:39,200 and think about what the problem is to even unblock those agents 264 00:14:39,200 --> 00:14:41,080 first. So yeah, that's the overview 265 00:14:41,080 --> 00:14:44,680 there. That's crazy, like 7-7 to 8. 266 00:14:44,680 --> 00:14:48,400 What was it? Yeah, 78910 again, you have 267 00:14:48,400 --> 00:14:51,400 these agentic sessions running. You need to have enough projects 268 00:14:51,400 --> 00:14:54,200 under your scope to even give them that much work. 269 00:14:54,480 --> 00:14:55,960 And they have to kind of all be separate. 270 00:14:56,080 --> 00:14:59,560 You can say, hey, I'm going to make this feature that depends 271 00:14:59,560 --> 00:15:03,120 on something like Agent 2 can't work on something that Agent 1 272 00:15:03,120 --> 00:15:04,680 hasn't already finished. That's the problem. 273 00:15:04,920 --> 00:15:07,640 So the, you know, Agent One will have to, but we'll be kind of 274 00:15:07,640 --> 00:15:12,480 stuck in the like feature, a complex feature development flow 275 00:15:12,480 --> 00:15:15,160 where it does the the one thing and then it'll have to do the 276 00:15:15,240 --> 00:15:17,440 the follow up on that maybe in a different code change. 277 00:15:17,880 --> 00:15:22,000 But again, if you are expanding into new verticals, trying new 278 00:15:22,000 --> 00:15:24,520 features, you can just take up a completely new work stream. 279 00:15:24,520 --> 00:15:27,280 It doesn't depend on anything previous and just have that be 280 00:15:27,280 --> 00:15:30,360 your second agent. So 8910, yeah, ambitious. 281 00:15:30,440 --> 00:15:32,800 You're probably doing not necessarily coding tasks at that 282 00:15:32,800 --> 00:15:34,320 point. This is going to be like your 283 00:15:34,320 --> 00:15:36,920 rewrite, you know, documentation updates or whatever else you're 284 00:15:36,920 --> 00:15:39,920 trying to do to clean up your code base or migrations. 285 00:15:40,760 --> 00:15:44,960 But yeah, certainly possible. It's just very small sort of 286 00:15:45,600 --> 00:15:47,040 cohort that people so far can do it. 287 00:15:47,480 --> 00:15:50,200 How does a roll out look like? Because I could imagine that not 288 00:15:50,320 --> 00:15:53,960 each engineer is like always on the cutting edge of things and I 289 00:15:53,960 --> 00:15:57,920 was keen to try out new stuff. Sometimes you want to just get 290 00:15:57,920 --> 00:16:01,200 your work done. How does a roll out look like to 291 00:16:01,200 --> 00:16:03,440 really get everyone on board? Yeah. 292 00:16:04,560 --> 00:16:07,720 So and how do you overcome like when people are hesitant to try 293 00:16:07,720 --> 00:16:09,520 new stuff? Yeah, yeah, yeah, for sure. 294 00:16:11,000 --> 00:16:14,600 So the roll outs it it they largely follow the adoption 295 00:16:14,600 --> 00:16:18,800 curve for various products. It's sort of a general concept 296 00:16:18,800 --> 00:16:22,720 of you've got these early adopters that jump on the 297 00:16:22,720 --> 00:16:23,880 product. If they know that it's good, 298 00:16:23,880 --> 00:16:25,840 they usually are plugged in themselves. 299 00:16:26,720 --> 00:16:29,920 So they, you know, follow like Twitter sentiment and whatever X 300 00:16:29,920 --> 00:16:34,640 sentiment and they come to us to ask for the product, like, Hey, 301 00:16:34,640 --> 00:16:36,560 when can I use this? So that's pretty good. 302 00:16:36,560 --> 00:16:42,000 That's essentially autonomous, you know, adoption, what is it? 303 00:16:42,040 --> 00:16:46,240 Organic adoption? And then once you get into that 304 00:16:46,240 --> 00:16:50,960 board, we look at if the results are good, you look at posts. 305 00:16:51,120 --> 00:16:54,680 Well, we, we do a multi channel reach out so you can do, you 306 00:16:54,680 --> 00:16:58,440 know, you have Slack channels for informing engineers. 307 00:16:58,720 --> 00:17:02,000 You have sort of show and tell sessions. 308 00:17:02,720 --> 00:17:06,680 If, if the product is really starting to roll, we try to do, 309 00:17:06,800 --> 00:17:09,319 you know, an hour of, hey, what's my workflow like? 310 00:17:09,960 --> 00:17:12,000 You essentially invite the whole engineering force into that 311 00:17:12,119 --> 00:17:15,119 whole meeting and have an engineer that's a power user 312 00:17:15,119 --> 00:17:17,880 show off what they've been doing to inspire and sort of show 313 00:17:17,880 --> 00:17:19,599 that, hey, it's easy to set up, you can do it. 314 00:17:20,440 --> 00:17:23,440 And we see that a lot of people follow that as if it was 315 00:17:23,440 --> 00:17:25,119 essentially a workshop because they're like, yeah, you know, 316 00:17:25,119 --> 00:17:26,839 this looks good to me. I want to ship more features. 317 00:17:27,319 --> 00:17:30,920 You know, productive workshops work well as well because 318 00:17:30,920 --> 00:17:32,960 that's, again, the same form factor, just a little more 319 00:17:32,960 --> 00:17:36,600 enforced than it in person. So it helps with, like, if you 320 00:17:36,600 --> 00:17:39,160 have multiple offices, you set up a workshop for a specific 321 00:17:39,160 --> 00:17:41,080 topic. I remember we had ones for 322 00:17:41,080 --> 00:17:45,680 cursor, not for Copilot before that, and then for a Cloud code 323 00:17:45,680 --> 00:17:47,920 after that. So, you know, whenever a new 324 00:17:48,080 --> 00:17:50,400 product comes along and people are interested, you set up a 325 00:17:50,400 --> 00:17:54,160 workshop and you do essentially the same thing. 326 00:17:54,160 --> 00:17:56,920 It's like, hey, here's a demo, perhaps try to create a feature 327 00:17:56,920 --> 00:17:59,000 or, you know, fix a bug or something through its workflow. 328 00:17:59,360 --> 00:18:05,240 And then you have an sort of an interactive audience to present 329 00:18:05,240 --> 00:18:07,880 your flow where someone can say, hey, you know, I'm having 330 00:18:07,880 --> 00:18:10,160 issues. And then you just walk over, you 331 00:18:10,280 --> 00:18:12,320 know, unblock them and and continue on. 332 00:18:13,760 --> 00:18:15,320 So yeah. And then Slack announcements, as 333 00:18:15,320 --> 00:18:19,840 I mentioned, you have enablement channels, so that and we have, 334 00:18:19,840 --> 00:18:25,080 you know, emails and Slack sort of news channels that you that 335 00:18:25,080 --> 00:18:29,000 people subscribe to. And then on top of that, I think 336 00:18:29,000 --> 00:18:32,280 we also have sort of an engineer lead culture, which is quite 337 00:18:32,280 --> 00:18:36,240 nice, where you have an engineer engineering lead, like engineer 338 00:18:37,040 --> 00:18:40,440 post in one of those channels to like signal boost again, like, 339 00:18:40,440 --> 00:18:43,080 hey, I stand behind this. I've I've checked this out. 340 00:18:43,240 --> 00:18:46,240 So that helps bring credibility to the actual tool and help 341 00:18:46,240 --> 00:18:48,520 people be more comfortable with using and trying it. 342 00:18:48,600 --> 00:18:51,440 Yeah. Not every company has that Uber 343 00:18:51,440 --> 00:18:53,880 engineering culture. Like do you have any tips and 344 00:18:53,880 --> 00:18:57,560 tricks on how to like engage engineers from other fields, 345 00:18:57,560 --> 00:19:02,160 from other companies, maybe larger organisations to really 346 00:19:02,400 --> 00:19:06,320 keep up with all the new tools and new techniques that are 347 00:19:06,320 --> 00:19:07,800 coming out? Yeah, yeah, yeah, for sure. 348 00:19:08,280 --> 00:19:11,520 I think a really powerful tool is having ambassadors for any 349 00:19:12,280 --> 00:19:14,960 sort of topic, not necessarily even in tech. 350 00:19:15,400 --> 00:19:19,520 And because the, the, the person itself that's an ambassador is 351 00:19:19,640 --> 00:19:24,480 likely to be motivated to, you know, show they, they believe in 352 00:19:24,480 --> 00:19:26,680 whatever that they're an ambassador for. 353 00:19:27,120 --> 00:19:31,720 And that really helps translate to the people that might not 354 00:19:31,840 --> 00:19:35,200 know about the tool or that you, you know, might want to have 355 00:19:35,200 --> 00:19:37,800 listened to to that reasoning there. 356 00:19:39,240 --> 00:19:41,920 So, yeah, essentially singling out because every single 357 00:19:41,920 --> 00:19:45,600 organization will have a more, a set of more like power users. 358 00:19:46,440 --> 00:19:49,640 You know, it will depend, the distribution will vary, but 359 00:19:49,720 --> 00:19:52,560 you'll have people that are like have everything set up and 360 00:19:52,560 --> 00:19:56,200 perhaps, you know, in your org or the power users, someone that 361 00:19:56,280 --> 00:19:58,520 just set up Claude code or maybe it's cursor. 362 00:19:59,400 --> 00:20:02,920 That's OK because you can use their, you know, if there's 363 00:20:02,960 --> 00:20:06,560 enthusiastic about the tool, you can leverage that to broadcast 364 00:20:06,560 --> 00:20:09,440 out to a more broader audience again, by just like creating 365 00:20:09,440 --> 00:20:13,960 perhaps a meeting show and tell. If it's a small enough company 366 00:20:13,960 --> 00:20:16,440 you can also get by with physical sessions again where 367 00:20:16,440 --> 00:20:19,240 you do a presentation to like the whole office or something 368 00:20:20,360 --> 00:20:22,760 and just show off like hey this is pretty easy to set up. 369 00:20:23,320 --> 00:20:27,560 I've got it working and it works a lot faster now than what I had 370 00:20:27,560 --> 00:20:30,280 before. Yeah, that's the initial 371 00:20:30,280 --> 00:20:32,800 thoughts. I, I wonder how you manage costs 372 00:20:32,800 --> 00:20:35,600 at Uber because like with all the different tools, it might 373 00:20:35,600 --> 00:20:38,600 get confusing. Like every tool has its certain 374 00:20:38,600 --> 00:20:42,080 pricing models and all of that. And I I. 375 00:20:42,080 --> 00:20:45,800 Think there's like a a? Tool life cycle where it gets 376 00:20:45,800 --> 00:20:48,640 hyped, it gets used, but then there's another tool coming 377 00:20:48,640 --> 00:20:51,880 along and you have a certain percentage of people like using 378 00:20:51,880 --> 00:20:55,080 the old one and like growing percentage using the new one. 379 00:20:55,320 --> 00:20:58,800 Like how do you manage costs with all your different tools 380 00:20:58,800 --> 00:21:00,480 that you set up? Yeah, yeah, for sure. 381 00:21:00,520 --> 00:21:06,400 The that is a real operational problem you have. 382 00:21:07,560 --> 00:21:10,400 Well, you know the the companies that produce the tools have the 383 00:21:10,400 --> 00:21:13,000 incentive to just give you a bunch of seats and keep charging 384 00:21:13,000 --> 00:21:15,680 for them. There is a challenge in if 385 00:21:15,680 --> 00:21:20,040 someone stops using a tool, how do you set up sort of proactive 386 00:21:20,040 --> 00:21:23,320 seat pruning and whatever else you can do to like take away the 387 00:21:23,320 --> 00:21:26,240 seat from the user, give it to someone else that maybe needs it 388 00:21:27,920 --> 00:21:31,000 and that that's that's some work you might need to do for smaller 389 00:21:31,000 --> 00:21:34,000 scale companies you can just get away with manually marking like 390 00:21:34,160 --> 00:21:36,360 disabling users. I don't think there's a problem 391 00:21:36,440 --> 00:21:40,400 with with that at all. The the key though, is if you 392 00:21:40,400 --> 00:21:43,560 have some tools provide like adoption metrics and you can 393 00:21:43,560 --> 00:21:45,920 see, you know, oh, you know, this user is using mostly 394 00:21:47,040 --> 00:21:50,920 whatever cursor or clod you can make informed decisions about 395 00:21:50,920 --> 00:21:55,720 the rest it I don't, I don't think based off of our stats 396 00:21:55,720 --> 00:21:59,760 that a lot of people do multimodal synchronous code 397 00:21:59,760 --> 00:22:01,680 editing. So like they, they will choose 398 00:22:01,720 --> 00:22:05,880 for the back background editing, they might pick, you know, and 399 00:22:06,320 --> 00:22:08,200 an interface. And for local synchronous flows, 400 00:22:08,200 --> 00:22:11,280 they pick an interface. And we had cases before where 401 00:22:11,280 --> 00:22:14,880 people, let's say from that prefer the Intellij ecosystem, 402 00:22:15,200 --> 00:22:19,800 pick a, let's say, pick up cursor just because the AI 403 00:22:19,880 --> 00:22:21,600 orchestrator is so good in cursor. 404 00:22:21,600 --> 00:22:24,720 So this was like a year ago or something, but they still want 405 00:22:24,720 --> 00:22:29,920 to have their main ID present. So this is kind of a bifurcated 406 00:22:29,920 --> 00:22:31,240 flow. It's not suboptimal. 407 00:22:32,120 --> 00:22:35,240 But then you have to pay for the two seats because again, people, 408 00:22:35,520 --> 00:22:36,920 that's what they find most useful. 409 00:22:37,160 --> 00:22:39,760 And we're trying to, you know, call us into a single more 410 00:22:39,800 --> 00:22:42,240 unified workflow where you don't have to contact switch. 411 00:22:42,240 --> 00:22:44,600 You've got your favorite orchestrator, A genetic 412 00:22:44,600 --> 00:22:49,320 orchestrator available. But it will inevitably lead to 413 00:22:49,320 --> 00:22:55,160 these overlaps. And yeah, so see pruning usage 414 00:22:55,160 --> 00:22:58,880 metrics tools to expose that depends on where you get your 415 00:22:58,880 --> 00:23:01,120 inference from. You'll see what who's using what 416 00:23:02,720 --> 00:23:09,480 in terms of yeah token, sort of what was it the the. 417 00:23:10,800 --> 00:23:13,240 If it's an outcome based outcome pricing. 418 00:23:14,040 --> 00:23:17,720 Well, outcome based is, is we haven't really gotten into that 419 00:23:17,720 --> 00:23:19,120 yet. The industry isn't yet asking 420 00:23:19,120 --> 00:23:23,240 for prices or like a cut of every single code change or 421 00:23:23,240 --> 00:23:26,880 something code line. I I think that that will be kind 422 00:23:26,880 --> 00:23:28,400 of damage. I mean, good for the inference 423 00:23:29,120 --> 00:23:31,000 company. I was trying to lean towards 424 00:23:31,160 --> 00:23:35,160 whether you allow engineers to spend at infinitum essentially, 425 00:23:35,520 --> 00:23:38,320 or whether you put a cap on their spend. 426 00:23:38,320 --> 00:23:40,280 I think there's a strong case for letting them spend on 427 00:23:40,280 --> 00:23:45,640 infinitum, but with perhaps what we found works as manual little 428 00:23:45,720 --> 00:23:49,440 checkpoints like saying, Hey, you've spent X, you know, 429 00:23:49,440 --> 00:23:52,360 dollars this month. Are you sure can? 430 00:23:52,400 --> 00:23:55,000 Can your manager sign off? Are you like not running in a 431 00:23:55,000 --> 00:23:57,040 circle, like just doing random crap? 432 00:23:57,680 --> 00:24:00,040 Are you using it productively? If yes, go go right ahead. 433 00:24:00,040 --> 00:24:01,280 You know, we'll support your usage. 434 00:24:01,840 --> 00:24:04,200 So that's a nice little speed bump that doesn't stop the 435 00:24:04,200 --> 00:24:08,200 engineer, but provides us a bit more or notifies the engineer of 436 00:24:08,200 --> 00:24:10,680 like, hey, B Cognizant because they might be running things 437 00:24:10,680 --> 00:24:12,960 that they don't even know are running in the background. 438 00:24:14,240 --> 00:24:14,960 Interesting. Yeah. 439 00:24:15,840 --> 00:24:19,200 For how long have you been doing this at Uber? 440 00:24:20,960 --> 00:24:28,080 We so from the very initial, like I think Copilot was the 441 00:24:28,400 --> 00:24:32,600 initial impetus of all of these coding assistants we had. 442 00:24:32,880 --> 00:24:34,960 That's when we started, we enabled Copilot really quickly 443 00:24:35,040 --> 00:24:38,040 and then we found that people, well, the initial Copilot wasn't 444 00:24:38,040 --> 00:24:42,680 really that much that capable with GBD 3 or what was it? 445 00:24:43,240 --> 00:24:47,600 Completions was one model codecs, but Chad was I think GPD 446 00:24:47,680 --> 00:24:53,440 3 or 3.5. And we found that people didn't 447 00:24:53,440 --> 00:24:57,120 really trust it yet. We had this big sort of hill to 448 00:24:57,120 --> 00:25:00,920 climb of like people not wanting anything to do with AI code. 449 00:25:00,920 --> 00:25:02,120 They're like, yeah, this is crappy. 450 00:25:02,720 --> 00:25:07,720 But again, through outside influence or sort of the, the 451 00:25:07,840 --> 00:25:10,480 overtone window shifting towards, Oh yeah, this might be 452 00:25:10,480 --> 00:25:14,320 producing good quality code that has largely been resolved and 453 00:25:14,320 --> 00:25:17,360 everyone's now on boarded. And we have like 95% of users 454 00:25:17,360 --> 00:25:19,720 using some kind of AI tool to augment their development 455 00:25:19,880 --> 00:25:26,360 workflow, which is nice. But yeah, the, the, the start, I 456 00:25:26,360 --> 00:25:30,400 think we kind of jumped really early onto the Copilot train and 457 00:25:30,400 --> 00:25:34,840 then we built like we had a big testing glut and we needed a 458 00:25:34,840 --> 00:25:38,000 testing tool that was capable. Copilot wasn't able to provide 459 00:25:38,040 --> 00:25:41,440 nowhere close to the amount of identity capability that we 460 00:25:41,440 --> 00:25:42,960 needed. So we built out a custom product 461 00:25:42,960 --> 00:25:47,880 built on a custom line graph graph with multiple sub agents 462 00:25:49,120 --> 00:25:52,080 or sort of a mixture of experts before that became a really a, 463 00:25:52,080 --> 00:25:57,160 a, a mainstream concept that's even built into the foundational 464 00:25:57,160 --> 00:26:00,040 agentic systems. That it was like what, 2-3 years 465 00:26:00,040 --> 00:26:01,160 ago? Yeah, yeah. 466 00:26:01,160 --> 00:26:07,120 It was around the time of the Copilot initial release and. 467 00:26:07,240 --> 00:26:10,280 Because where I was going with the questions, like I, I really 468 00:26:10,280 --> 00:26:13,120 wanted to know what was the most surprising thing that came up 469 00:26:13,160 --> 00:26:16,520 during that time? Like did you encounter any like 470 00:26:16,640 --> 00:26:20,320 super surprising situation that you didn't anticipate? 471 00:26:22,920 --> 00:26:26,400 I think. We, we just the, the, IT wasn't 472 00:26:26,400 --> 00:26:31,200 so much as a surprise, but the realisation that the genetic 473 00:26:31,200 --> 00:26:35,880 assistance back in the day, you must build them as a, a genetic 474 00:26:35,880 --> 00:26:39,480 assistant provider in a way that makes them extreme generalists. 475 00:26:39,680 --> 00:26:43,600 So you, you know, you, you have, we didn't even have command 476 00:26:43,600 --> 00:26:45,600 execution back then. It was just code editing. 477 00:26:45,920 --> 00:26:50,160 But even then you have this whole gamut of users across a 478 00:26:50,560 --> 00:26:54,640 whole range of open source and closed source code bases, you 479 00:26:54,640 --> 00:26:57,800 know, dozens of languages and you need to be able to operate 480 00:26:57,800 --> 00:27:00,560 in all of them at the same time. We had this sort of like, hey, 481 00:27:00,880 --> 00:27:04,640 we can build some really capable deterministically enhanced 482 00:27:04,640 --> 00:27:07,320 flows. So you can imagine we actually 483 00:27:07,320 --> 00:27:10,600 went that direction with Claude Code now having skills as an 484 00:27:10,600 --> 00:27:14,000 example where you can plug in a script and the script actually 485 00:27:14,000 --> 00:27:16,760 does some deterministic action for the agent and it's able to 486 00:27:16,760 --> 00:27:19,840 like launch it and leverage it and then act on the result. 487 00:27:21,320 --> 00:27:27,280 That direction we realized very early is would be moving forward 488 00:27:27,280 --> 00:27:31,920 because again, for workflows become competent and especially 489 00:27:31,920 --> 00:27:34,880 within like a larger scale company, you need to plug in 490 00:27:34,880 --> 00:27:36,960 some deterministic flows. And that's what we were able to 491 00:27:36,960 --> 00:27:39,960 do with the land graph graph that I was talking about. 492 00:27:40,360 --> 00:27:42,760 So you can do like crazy, you know, you plug in your build 493 00:27:42,760 --> 00:27:46,240 system, you can customize to for it to fit your build system 494 00:27:46,240 --> 00:27:47,760 exactly. So we can do, you know, hundreds 495 00:27:47,760 --> 00:27:50,600 of parallel builds at the same time as an example, instead of 496 00:27:50,600 --> 00:27:52,880 A1 single command. And that's like claw can't even 497 00:27:52,880 --> 00:27:54,560 do that today unless you build some skill. 498 00:27:54,720 --> 00:27:58,160 I don't think it will likely ever, well, not ever. 499 00:27:58,160 --> 00:28:01,920 I will not say that, but. Ever is a very strong word here. 500 00:28:01,920 --> 00:28:02,640 Very strong. Word. 501 00:28:02,640 --> 00:28:03,680 It's probably going to be able to. 502 00:28:03,720 --> 00:28:06,240 Do it in a year. But what I'm trying to say is in 503 00:28:06,480 --> 00:28:09,040 like cloud still runs, even if you do parallel agents, it still 504 00:28:09,040 --> 00:28:10,600 runs same build, the same build commands. 505 00:28:10,760 --> 00:28:13,280 You'll need to provide some skills for it to like understand 506 00:28:13,280 --> 00:28:16,840 how to even like finagle your build system into running 100 507 00:28:16,840 --> 00:28:19,440 different permutations of the same file to like try different 508 00:28:19,440 --> 00:28:23,280 variants, stuff like that. So extreme agentic programming 509 00:28:24,400 --> 00:28:28,880 is very custom and the bifurcation that I was trying to 510 00:28:28,880 --> 00:28:31,720 explain to that we realized this, the generalist assistance 511 00:28:32,120 --> 00:28:35,720 will not like automate this away like this will never happen. 512 00:28:35,840 --> 00:28:38,880 We need to build this these foundational building blocks 513 00:28:38,880 --> 00:28:42,000 that, you know, plug into our build system and know how to 514 00:28:42,000 --> 00:28:44,560 operate at Uber. And that's what we're continuing 515 00:28:44,560 --> 00:28:48,160 with. I think over time we found that 516 00:28:48,360 --> 00:28:51,480 the agentic like the speed isn't that much of A bottleneck. 517 00:28:51,640 --> 00:28:54,400 So the agentic generalist assistant is kind of good enough 518 00:28:54,400 --> 00:28:56,560 for a lot of things, but it needs these tendrils. 519 00:28:56,560 --> 00:28:59,240 It needs a tendril to reach into like the build system. 520 00:28:59,480 --> 00:29:02,640 It needs a tendril to be able to understand how what conventions 521 00:29:02,640 --> 00:29:05,320 we have with Uber or like search docs, stuff like that. 522 00:29:06,000 --> 00:29:11,160 Being for it to like interface with what Uber is all about and 523 00:29:11,160 --> 00:29:12,720 our engineering practices is very important. 524 00:29:13,520 --> 00:29:15,680 But that surface area is again now plugged by things like 525 00:29:15,680 --> 00:29:18,600 skills, which are easy to develop, you know, improve. 526 00:29:19,520 --> 00:29:25,240 That's, that's that. The one question that what that 527 00:29:25,240 --> 00:29:31,320 really interests me is like did the adaptation of agents and 528 00:29:31,320 --> 00:29:35,560 agent coding change the way engineering teams are 529 00:29:35,560 --> 00:29:38,560 structured? Like did you see a change in how 530 00:29:38,560 --> 00:29:41,440 the team's structured, how the team is set up, how teams 531 00:29:41,440 --> 00:29:46,200 behave, or like cross team initiatives get managed? 532 00:29:46,560 --> 00:29:52,520 Yeah, that's a good question. So if we call back to earlier in 533 00:29:52,520 --> 00:29:55,200 this conversation where I said you needed to expand the amount 534 00:29:55,200 --> 00:29:57,920 of threads and features are parallel working, working on 535 00:29:57,920 --> 00:30:04,760 parallel at the same time, this lends itself really nicely to 536 00:30:05,400 --> 00:30:09,280 more efforts that are essentially more efforts between 537 00:30:09,280 --> 00:30:10,720 more teams. So we got a bit more 538 00:30:10,720 --> 00:30:14,240 connectivity. You've got people from a team 539 00:30:14,800 --> 00:30:18,200 joining more virtual teams now, at least from what I observed, 540 00:30:18,200 --> 00:30:20,480 meaning a virtual team is like, they're not part of a, a 541 00:30:20,800 --> 00:30:24,120 different team, but they're part of an effort that's like outside 542 00:30:24,120 --> 00:30:26,400 of their team potentially, but needs the involvement, 543 00:30:26,520 --> 00:30:31,760 involvement of their team. So then what you get is someone 544 00:30:31,760 --> 00:30:34,800 dedicates, let's say 10 or 20% of their time to a specific V 545 00:30:34,800 --> 00:30:37,080 team. But in effect, their 546 00:30:37,080 --> 00:30:41,640 contribution to that effort is now like 80 or 90% of what it 547 00:30:41,640 --> 00:30:45,760 could have been if they went in like essentially full time on to 548 00:30:45,760 --> 00:30:48,120 that whole effort, but before coding assistance. 549 00:30:48,520 --> 00:30:51,280 So if you imagine like they can deliver with the leverage that 550 00:30:51,280 --> 00:30:55,000 coding assistance offer, they can deliver like full time or 551 00:30:55,000 --> 00:30:59,480 even more as we get into the, the more leveraged work streams, 552 00:31:00,000 --> 00:31:03,640 you can deliver more that you could have before if you've like 553 00:31:03,640 --> 00:31:06,080 full time joined that team. So that's awesome because again, 554 00:31:06,080 --> 00:31:08,480 you, you have these like whatever amount of E teams you 555 00:31:08,480 --> 00:31:11,960 can manage mentally and you're pushing everything forward 556 00:31:11,960 --> 00:31:13,840 because it's really not you're working, you're managing a bunch 557 00:31:13,840 --> 00:31:16,360 of agents. So you have you're like an 558 00:31:16,360 --> 00:31:20,520 engineering tech lead, I guess that has his fingers and a bunch 559 00:31:20,520 --> 00:31:23,320 of different efforts. But you've got these virtual, 560 00:31:23,520 --> 00:31:26,000 you know, teammates that actually like the the agents 561 00:31:26,600 --> 00:31:29,480 that deliver on those because someone needs to write the code 562 00:31:29,480 --> 00:31:30,720 and validate the code ultimately. 563 00:31:30,920 --> 00:31:35,600 So that that's the sort of impact on the organization that 564 00:31:35,600 --> 00:31:37,960 we see so far. Before we close the episode, I'm 565 00:31:37,960 --> 00:31:41,240 really curious if if you have any tips and tricks for our 566 00:31:41,240 --> 00:31:45,120 listeners to really stay at the edge of tech and do not miss out 567 00:31:45,120 --> 00:31:47,360 on any opportunities on new things that are coming up. 568 00:31:47,600 --> 00:31:51,680 Like, what does your workflow look like when it comes to 569 00:31:51,680 --> 00:31:54,520 really being at the forefront of what is coming out? 570 00:31:54,720 --> 00:32:00,120 Yeah, yeah, for sure. So I personally have I, I, I'm a 571 00:32:00,560 --> 00:32:03,520 avid believer in the personal assistant space that's sort of 572 00:32:03,520 --> 00:32:06,320 popping up right this moment. So you've got open claw or nano 573 00:32:06,320 --> 00:32:10,320 claw or whatever else. It's non trivial to set it up in 574 00:32:10,320 --> 00:32:13,000 a secure way. So I will not recommend any 575 00:32:13,240 --> 00:32:15,560 anyone to do it because if you give it access to your whole 576 00:32:15,560 --> 00:32:18,280 laptop, there's a lot of implications that you. 577 00:32:18,600 --> 00:32:20,000 Don't want. Don't, don't, don't. 578 00:32:20,000 --> 00:32:22,200 Want to do that? No, no, don't do that because 579 00:32:22,200 --> 00:32:24,000 the Mac mini is already expensive enough. 580 00:32:24,000 --> 00:32:24,960 Don't. Well. 581 00:32:25,440 --> 00:32:28,920 So yeah, yeah, you either have to like completely separate it 582 00:32:28,920 --> 00:32:33,120 out, run it on a VPSI know companies are now coming up 583 00:32:33,120 --> 00:32:35,960 with, you know, isolated environments to run these steps. 584 00:32:35,960 --> 00:32:37,760 Yeah. But then do you, do you have the 585 00:32:37,760 --> 00:32:40,240 data on the machine that it takes to really get good 586 00:32:40,240 --> 00:32:41,000 results? Well, well. 587 00:32:41,240 --> 00:32:45,040 The good thing is you can have it like be a super consumer of 588 00:32:45,040 --> 00:32:48,040 public streams. So things like let's say you 589 00:32:48,040 --> 00:32:51,040 have whatever the amount of RSS feeds, 50 RSS feeds you're 590 00:32:51,040 --> 00:32:53,840 interested in, kinda like Hacker News. 591 00:32:53,880 --> 00:32:57,240 That's essentially you don't need the login session for that. 592 00:32:57,240 --> 00:32:58,120 You don't need anything for that. 593 00:32:58,560 --> 00:33:00,920 If you just run it on like AVPS that's not connected to any of 594 00:33:00,920 --> 00:33:03,480 your data at all, it will be able to fetch those sort of 595 00:33:03,840 --> 00:33:07,280 surface level things. All of the labs just give it let 596 00:33:07,280 --> 00:33:10,880 it research what to research. That's sort of a meta level 597 00:33:10,880 --> 00:33:12,640 thing. It's like find all the top labs, 598 00:33:12,640 --> 00:33:15,600 you know, whatever the, you know, open AI anthropic Gemini 599 00:33:16,920 --> 00:33:22,240 and look at that matrix of things like let it run, let's 600 00:33:22,240 --> 00:33:24,320 say every morning or something if you want to be really plugged 601 00:33:24,320 --> 00:33:27,640 in and let let it like send you a report over whatever the 602 00:33:27,640 --> 00:33:30,600 interface you have. And that's a really good way I 603 00:33:30,600 --> 00:33:36,320 found because it can also like you would need to set up like 604 00:33:36,680 --> 00:33:40,080 tokens and API tokens for let's say Twitter analysis, like 605 00:33:40,160 --> 00:33:43,040 popular sentiment analysis. But you could do that. 606 00:33:43,040 --> 00:33:45,160 And then you have this whole like the whole thing that I 607 00:33:45,160 --> 00:33:46,240 mentioned that started the episode. 608 00:33:46,240 --> 00:33:51,560 It's like all vibes and you sort of yeah, like vibe based 609 00:33:51,880 --> 00:33:54,640 understanding. You get that in report form in 610 00:33:54,640 --> 00:33:57,080 the morning, which is really powerful and you can take what 611 00:33:57,080 --> 00:33:59,360 you want from it. You usually get all the news 612 00:33:59,360 --> 00:34:02,200 like the hour delta in that report. 613 00:34:02,200 --> 00:34:04,320 So I always see them. I never see anything missed 614 00:34:04,440 --> 00:34:07,280 because the report is kind of large based on the amount of 615 00:34:07,280 --> 00:34:08,960 things that I added to for it to check out. 616 00:34:10,000 --> 00:34:13,000 But yeah, that's probably the top thing that I found. 617 00:34:13,000 --> 00:34:15,120 It's like the most helpful. Otherwise, you just make a list 618 00:34:15,120 --> 00:34:18,360 and you go like once per week, like hack your news, whatever 619 00:34:18,880 --> 00:34:22,199 Google Scholar papers pop up at the time that are important to 620 00:34:22,199 --> 00:34:23,639 read. Yeah. 621 00:34:25,120 --> 00:34:26,960 Thank you so much for coming to the episode. 622 00:34:27,120 --> 00:34:30,960 I learned a metric shit ton. It was really interesting. 623 00:34:30,960 --> 00:34:33,239 I was really glad you're here. Thank you so much. 624 00:34:33,440 --> 00:34:36,199 And yeah, let's talk to you. Talk again very soon. 625 00:34:36,400 --> 00:34:39,639 Thank you so much for having me. Thanks everyone for making it to 626 00:34:39,639 --> 00:34:41,880 the end of this episode. I hope you enjoyed the 627 00:34:41,880 --> 00:34:44,040 conversation with Amachas as much as I did. 628 00:34:44,040 --> 00:34:46,480 If you're digging the podcast, do me a huge favour. 629 00:34:46,760 --> 00:34:49,400 Hit the subscribe button, drop a like and if you're feeling extra 630 00:34:49,400 --> 00:34:51,920 generous, leave a revenue. It really helps us get these 631 00:34:51,920 --> 00:34:53,600 conversations in front of more. People. 632 00:34:53,880 --> 00:34:56,880 We got some amazing guests lined up so you definitely don't want 633 00:34:56,880 --> 00:34:59,440 to miss what's coming next. So then see you soon. 634 00:34:59,680 --> 00:35:00,040 Bye.