1 00:00:00,040 --> 00:00:02,200 This episode is brought to you by Beam OPS. 2 00:00:02,360 --> 00:00:05,480 If your Elixir deployments are painful, or your infrastructure 3 00:00:05,480 --> 00:00:07,920 is a mess, or you just need experience hands to help you 4 00:00:07,920 --> 00:00:11,840 scale, these are your people. Beam OPS specializes in scalable 5 00:00:11,840 --> 00:00:14,800 Elixir development, automated deployment pipelines, and 6 00:00:14,800 --> 00:00:18,000 infrastructure migrations. They will help you work smarter 7 00:00:18,040 --> 00:00:21,480 and ship faster. As authors of the Pragprog book 8 00:00:21,560 --> 00:00:24,480 Engineering Elixir Applications, they know their stuff. 9 00:00:24,720 --> 00:00:30,720 Check them out at beamops.co.uk. Secure your Elixir apps with 10 00:00:30,720 --> 00:00:33,440 paraxial IO. From code analysis and bot 11 00:00:33,440 --> 00:00:36,040 defense to dependency checks, they've got you covered. 12 00:00:36,200 --> 00:00:39,000 Elixir first design with GitHub integration. 13 00:00:39,120 --> 00:00:46,240 Elevate your security at paraxial dot IO testing. 14 00:00:50,120 --> 00:00:52,440 What's up? I'm Jacob Litzio, your Elixir 15 00:00:52,440 --> 00:00:55,360 mentor, and welcome to another exciting episode of the Elixir 16 00:00:55,360 --> 00:00:57,600 Mentor Podcast. This is where we discuss 17 00:00:57,640 --> 00:01:00,000 everything related to Elixir, from interviews with 18 00:01:00,000 --> 00:01:02,680 enthusiasts, pioneers in the community, to innovative 19 00:01:02,680 --> 00:01:05,720 projects and libraries shaping Elixir's future. 20 00:01:06,160 --> 00:01:09,840 Dave returns to the podcast this time to talk about founding TV 21 00:01:09,840 --> 00:01:14,160 Labs, where he's using Elixir, Lua and computer vision to test 22 00:01:14,160 --> 00:01:18,560 smart TV apps on real physical devices at scale. 23 00:01:18,560 --> 00:01:22,240 But I won't butcher too much of what you do, Dave, Welcome back. 24 00:01:22,280 --> 00:01:25,840 It's good to have you. And yeah, you want to give a 25 00:01:25,840 --> 00:01:28,600 brief introduction of who you are and what you're doing. 26 00:01:29,600 --> 00:01:32,800 Absolutely. First off, thank you very much 27 00:01:32,800 --> 00:01:35,240 for having me back on. It's exciting to be here. 28 00:01:36,200 --> 00:01:40,040 A little bit about me again, I'm Dave Lucia. 29 00:01:40,040 --> 00:01:44,880 I Co founded a company called TD Labs about 3 years ago. 30 00:01:45,280 --> 00:01:49,320 We're a smart TV and connected device testing platform. 31 00:01:50,600 --> 00:01:56,640 You can think of us as a virtual environment for any physical TV, 32 00:01:56,640 --> 00:02:01,160 smart TV that you can imagine. So if you have like an LGTV in 33 00:02:01,160 --> 00:02:04,840 your living room, we have a ton of those in our device lab. 34 00:02:04,840 --> 00:02:07,600 We virtualize them through the cloud and then allow you to 35 00:02:07,600 --> 00:02:11,280 interact with them through your browser through an Appium test 36 00:02:11,280 --> 00:02:16,120 script or using our drag and drop automation platform and 37 00:02:16,120 --> 00:02:20,800 have access to hundreds of different types of devices, all 38 00:02:20,800 --> 00:02:22,920 virtually. And that's all built in Elixir 39 00:02:22,920 --> 00:02:25,440 for the most part. And we're doing some pretty cool 40 00:02:25,440 --> 00:02:28,800 stuff when it comes to AI, computer vision, distributed 41 00:02:28,800 --> 00:02:30,960 systems, kind of all of the above. 42 00:02:31,680 --> 00:02:33,920 So yeah, that's good about me. Yeah, that's wild. 43 00:02:34,000 --> 00:02:38,600 I'm, I'm kind of curious what even got you in that space? 44 00:02:38,640 --> 00:02:42,040 Like physical devices and TV testing, Like what does that 45 00:02:42,040 --> 00:02:43,760 even? What's that look like? 46 00:02:44,800 --> 00:02:50,720 Yeah, We have to go back about 10 years or so to when I was at 47 00:02:50,720 --> 00:02:54,200 Bloomberg. So I worked at Bloomberg at the 48 00:02:54,200 --> 00:02:57,440 start of my career and originally was working on 49 00:02:57,440 --> 00:03:00,680 Bloomberg trade book, which is a trading platform writing a lot 50 00:03:00,680 --> 00:03:03,640 of C++. But I moved over to the web 51 00:03:03,640 --> 00:03:06,600 team. This is back in like circa 2014 52 00:03:07,480 --> 00:03:10,960 and started working on the Bloomberg business, Bloomberg 53 00:03:10,960 --> 00:03:16,120 Business Week relaunch in 2014 in Bloomberg Politics. 54 00:03:17,120 --> 00:03:20,000 And I met this guy there, Stephen Baldwin, who was a 55 00:03:20,000 --> 00:03:23,080 contractor. He had his own company called 56 00:03:23,080 --> 00:03:26,120 Econify and he was also subcontracting himself out to 57 00:03:26,120 --> 00:03:28,360 Bloomberg. And we became friendly and 58 00:03:28,880 --> 00:03:30,320 stayed in touch throughout the years. 59 00:03:30,360 --> 00:03:36,960 And I had now let's zoom forward to to 2023. 60 00:03:37,760 --> 00:03:42,560 I was at a start up called Milk Road and things weren't really 61 00:03:42,560 --> 00:03:45,080 working out there. And we, we started chatting 62 00:03:45,560 --> 00:03:50,320 after I departed and he's like, hey, I have this problem. 63 00:03:50,360 --> 00:03:55,960 We are working with a bunch of media companies and they're all 64 00:03:55,960 --> 00:03:59,520 trying to build smart TV apps and we're building these device 65 00:03:59,520 --> 00:04:03,160 labs, you know, just taking devices and pointing webcams at 66 00:04:03,160 --> 00:04:04,080 it. And he's like, we did this 67 00:04:04,080 --> 00:04:06,600 during the pandemic and it was really hard. 68 00:04:06,600 --> 00:04:10,480 And what I realized is there is a huge opportunity here to build 69 00:04:12,200 --> 00:04:14,520 a, a product around smart TV testing. 70 00:04:15,600 --> 00:04:19,079 And so I was like, this sounds like an awesome problem. 71 00:04:19,079 --> 00:04:23,080 We, we dug into it. You know, I spent a few months 72 00:04:23,600 --> 00:04:27,800 with him and our other Co founder Thomas building an MVP 73 00:04:27,800 --> 00:04:29,280 of what the product would be like. 74 00:04:29,280 --> 00:04:32,760 We had like, you know, just a little screen that that showed, 75 00:04:34,080 --> 00:04:36,360 you know, we're pointing a camera at ATV. 76 00:04:36,840 --> 00:04:42,560 And then we had web RTC going from the TV directly to the web 77 00:04:42,560 --> 00:04:45,040 browser. And you know, having that first 78 00:04:45,040 --> 00:04:47,960 MVP of being able to like click a button in your web browser and 79 00:04:47,960 --> 00:04:50,800 see the TV react to that. It was amazing. 80 00:04:51,520 --> 00:04:58,120 And we from there, we, we raised some, some money from friends 81 00:04:58,120 --> 00:05:00,800 and family and, and kind of started running at the problem. 82 00:05:01,720 --> 00:05:04,680 And Fast forward later, Fast forward about three years later, 83 00:05:04,680 --> 00:05:08,840 we've got a bunch of clients, some of the biggest media 84 00:05:08,840 --> 00:05:13,200 companies in the world using the product in anger, running 85 00:05:13,520 --> 00:05:16,160 thousands of tests on our platform per day. 86 00:05:16,280 --> 00:05:20,400 And yeah, it's, it's been a really interesting and deep, 87 00:05:20,600 --> 00:05:22,800 deep and wide problem to jump into. 88 00:05:22,840 --> 00:05:29,160 So in short, I I met someone 10 years ago, a friend, and we've 89 00:05:29,160 --> 00:05:33,440 been trying to work together for more than a decade and kind of 90 00:05:33,440 --> 00:05:38,240 the stars aligned timing wise and we kind of left our jobs and 91 00:05:38,240 --> 00:05:40,440 started a new thing. That's wild. 92 00:05:40,920 --> 00:05:44,600 Did you raise money before you left your job or did you leave 93 00:05:44,600 --> 00:05:48,160 your job before? No, I left, left the job, 94 00:05:48,640 --> 00:05:53,000 started consulting. Basically I split half my time 95 00:05:53,440 --> 00:05:56,760 billing like 20 hours a week consulting and the other half 96 00:05:56,960 --> 00:06:01,000 working on the MVP of TV Labs were probably about three months 97 00:06:01,000 --> 00:06:06,080 before we raised money. Did did the vision of TV labs? 98 00:06:06,200 --> 00:06:10,440 Has it changed drastically since like the MVP to today or? 99 00:06:11,840 --> 00:06:13,880 Same kind. Of more features. 100 00:06:14,880 --> 00:06:19,040 I think though, what's really changed in terms of the vision 101 00:06:19,040 --> 00:06:25,120 is just more of everything when it comes to capabilities. 102 00:06:25,120 --> 00:06:31,320 So we, we always have the vision of replacing QA on, on connected 103 00:06:31,320 --> 00:06:33,680 devices. So make it really, really easy 104 00:06:33,880 --> 00:06:37,600 to test on any, any type of device that you can imagine. 105 00:06:37,600 --> 00:06:45,000 ATV, the back of an airplane screen, you know, your Peloton, 106 00:06:45,000 --> 00:06:49,440 anything that has a screen on it that you might be able to, to 107 00:06:50,000 --> 00:06:51,840 watch some sort of streaming media app. 108 00:06:52,080 --> 00:06:54,680 We wanted to be the testing platform for doing that. 109 00:06:54,920 --> 00:06:56,720 And that's our still still our goal today. 110 00:06:57,240 --> 00:07:01,480 What I think has changed is while we've been building this 111 00:07:01,480 --> 00:07:05,200 business, AI has been evolving with it. 112 00:07:06,160 --> 00:07:09,120 So we've been using AI pretty much since the beginning. 113 00:07:09,760 --> 00:07:13,160 I think what was it like 2022 is when like the first interesting 114 00:07:13,160 --> 00:07:17,520 release of the GPT 3 came out. And so we're kind of in the 115 00:07:17,520 --> 00:07:19,920 early days of of experimenting with that. 116 00:07:20,400 --> 00:07:23,520 And what I think is really solidified in the past year is 117 00:07:23,520 --> 00:07:27,240 that those tools are becoming more and more core to our 118 00:07:27,240 --> 00:07:29,840 product, not just in terms of what we're offering to 119 00:07:29,840 --> 00:07:34,320 customers, but also our operations and how we, you know, 120 00:07:34,320 --> 00:07:37,640 interact with devices and how we maintain them and how we respond 121 00:07:37,640 --> 00:07:40,000 to issues. Kind of AI is sort of getting 122 00:07:40,000 --> 00:07:43,400 integrated into all of that. So I, I would say that's the the 123 00:07:43,400 --> 00:07:48,360 most, the biggest change that I, I've seen to our, our business 124 00:07:48,360 --> 00:07:54,600 is just kind of wanting to step into more domains and thinking 125 00:07:54,600 --> 00:07:58,520 that more is possible because of kind of everything that's 126 00:07:58,520 --> 00:08:03,120 changing in the world right now. Yeah, things are moving quick in 127 00:08:03,120 --> 00:08:06,840 the, the AI world, aren't they? I feel like I learned new things 128 00:08:06,840 --> 00:08:12,560 how people are using LLMS for like everyday workflows and not 129 00:08:12,560 --> 00:08:15,360 just coding anymore, right? It's like simplify things in 130 00:08:15,360 --> 00:08:18,280 your life using it or just processes you have in your 131 00:08:18,280 --> 00:08:21,080 business. But before, before we dive too 132 00:08:21,080 --> 00:08:23,800 deep there, because I think that'll be a good conversation. 133 00:08:24,360 --> 00:08:28,800 I'm I'm always curious like you're a you're a very technical 134 00:08:29,440 --> 00:08:32,799 founder, right And you did you Co found TV. 135 00:08:32,799 --> 00:08:36,200 Labs, Yeah, Yep. So there's actually three of us 136 00:08:36,200 --> 00:08:39,080 and we're all technical, which is actually a good and a bad 137 00:08:39,080 --> 00:08:41,520 thing. So like that's the worst thing 138 00:08:41,520 --> 00:08:46,760 ever to do, right? What did you, did you find it 139 00:08:46,760 --> 00:08:49,200 difficult to find clients at the beginning? 140 00:08:49,200 --> 00:08:52,080 And like what were your strategies and did one of you 141 00:08:52,080 --> 00:08:55,600 step into that role? Did you were just willing to be 142 00:08:55,600 --> 00:08:58,680 more of the marketing salesperson or did all of you 143 00:08:58,680 --> 00:09:02,280 want to build a product? Very good question. 144 00:09:02,280 --> 00:09:06,280 So there's three Co founders at TV Labs. 145 00:09:06,280 --> 00:09:12,640 It's myself, Thomas Wustenfeld, who is our CRDT chief research 146 00:09:12,640 --> 00:09:15,080 and development Czar. I'm still trying to figure out 147 00:09:15,080 --> 00:09:17,600 what his title means, but he wanted to be called the CRDT. 148 00:09:18,640 --> 00:09:21,320 He's effectively head of our hardware initiative. 149 00:09:22,440 --> 00:09:28,840 I can show you we have we develop our own PCBS here at TV 150 00:09:28,840 --> 00:09:29,880 labs there we. Go. 151 00:09:30,440 --> 00:09:33,200 Goes up like that. This is a very old version of 152 00:09:33,200 --> 00:09:35,960 it. And then we have Steven who is 153 00:09:35,960 --> 00:09:38,840 our CEO. And what's interesting is that 154 00:09:38,840 --> 00:09:45,000 the company that I met Steven through it's called Econify and 155 00:09:45,520 --> 00:09:49,680 there was a consultancy and they primarily consulted with media 156 00:09:49,680 --> 00:09:53,400 companies. And again, this, this product 157 00:09:53,400 --> 00:09:55,480 was kind of born out of that consultancy. 158 00:09:56,520 --> 00:10:02,000 So with TV Labs, we kind of had existing relationships through, 159 00:10:02,000 --> 00:10:06,080 you know, that prior business with, you know, all of these 160 00:10:06,080 --> 00:10:08,360 different media companies. And that's been really, really 161 00:10:08,360 --> 00:10:11,680 helpful for a brand new business, you know, trying to to 162 00:10:11,680 --> 00:10:15,520 get your foot into the door into these large media companies who 163 00:10:15,560 --> 00:10:17,840 you know, are not going to give the time of day to just a small 164 00:10:18,200 --> 00:10:19,680 random company they've never cared of it's. 165 00:10:19,880 --> 00:10:23,120 Easy to get your foot in the door for a large company, right? 166 00:10:24,360 --> 00:10:28,400 No, absolutely not. And that's where I think, you 167 00:10:28,440 --> 00:10:31,760 know, Steven stepping into that role of, you know, his 168 00:10:31,760 --> 00:10:35,040 background as an engineer, but he was also the, you know, half 169 00:10:35,040 --> 00:10:39,960 owner and CTO of Econify. He really spent a lot of his 170 00:10:39,960 --> 00:10:43,520 time on sales and building relationships. 171 00:10:44,400 --> 00:10:47,360 And you know, that's kind of the the role he sits in. 172 00:10:47,360 --> 00:10:50,320 He's very involved in the product and very involved in, 173 00:10:51,280 --> 00:10:54,120 you know, gets his hands dirty here and there on writing code 174 00:10:55,360 --> 00:10:56,840 or cursor writing code these days. 175 00:10:56,840 --> 00:11:02,800 But more so, it's like the, you know, fundraising, you know 176 00:11:03,200 --> 00:11:07,960 client outreach, you know, sales, you know, deal closing 177 00:11:07,960 --> 00:11:09,080 all the. Stuff you need that. 178 00:11:09,240 --> 00:11:11,360 You need that person right Otherwise. 179 00:11:11,520 --> 00:11:14,400 Yeah. And and you know, negotiating 180 00:11:14,400 --> 00:11:16,600 enterprise contracts is, is no joke. 181 00:11:16,960 --> 00:11:19,720 It's a whole different skill set and I'm sure. 182 00:11:19,720 --> 00:11:24,400 Yeah, so. Probably is horrible. 183 00:11:25,160 --> 00:11:27,480 Oh, yeah, yeah, it's brutal. It's brutal. 184 00:11:27,480 --> 00:11:31,080 Security reviews, you know, going through contracts. 185 00:11:31,080 --> 00:11:35,280 It's, it's, it's very different than, you know, writing code or 186 00:11:35,280 --> 00:11:39,120 working on a product. It's, it feels like, you know, 187 00:11:39,120 --> 00:11:42,440 business. So it's good to have that skill 188 00:11:42,440 --> 00:11:43,800 set on the on the leadership team. 189 00:11:43,800 --> 00:11:47,680 So I don't, this might be a dumb question, but I'm do you, do you 190 00:11:47,680 --> 00:11:51,760 sell a physical product to these companies or do you test for 191 00:11:51,760 --> 00:11:53,280 them in your locations? How does? 192 00:11:53,360 --> 00:11:54,880 What does that? How does that work? 193 00:11:55,640 --> 00:12:01,800 Very good question. So there's a lot of services out 194 00:12:01,800 --> 00:12:05,560 there where you kind of buy physical units, hardware that 195 00:12:05,560 --> 00:12:08,840 kind of run in your lab. That's not what we do. 196 00:12:09,560 --> 00:12:15,000 We effectively rent or lease out devices in a shared model. 197 00:12:15,000 --> 00:12:18,640 We have both a shared and dedicated model, But you know, 198 00:12:18,680 --> 00:12:22,120 what our technology allows for is you come onto our platform, 199 00:12:22,120 --> 00:12:25,600 you say, you know, I want a particular, you know, a 2022 200 00:12:25,600 --> 00:12:31,120 Roku 4K and our matchmaking engine, we'll, we'll find a 201 00:12:31,120 --> 00:12:35,280 device that that matches your request, initiates a session, 202 00:12:35,560 --> 00:12:38,640 and now you're directly communicating with that device 203 00:12:38,640 --> 00:12:41,920 over our platform. So all of our devices are hosted 204 00:12:41,920 --> 00:12:45,520 in our lab and it's only software, you know, everything 205 00:12:45,520 --> 00:12:47,200 is virtual that you're connecting through so. 206 00:12:47,720 --> 00:12:51,320 Wait, how many devices do you have in like your warehouse then 207 00:12:51,360 --> 00:12:54,840 or your facility? Well, yeah, right now I don't 208 00:12:54,840 --> 00:12:57,440 want to say on this call, but right now we're adding like on a 209 00:12:57,440 --> 00:13:00,440 month to month basis, we're adding hundreds of new devices. 210 00:13:00,720 --> 00:13:07,160 We've got, yeah, we've got a big, a big space in Brooklyn 211 00:13:07,160 --> 00:13:10,960 where a lot of our devices are held and then we have kind of 212 00:13:10,960 --> 00:13:14,880 smaller locations elsewhere. But our main office in Brooklyn 213 00:13:14,880 --> 00:13:18,120 is where we have kind of the the biggest footprint of devices. 214 00:13:18,120 --> 00:13:22,520 How do you, how do you like stay on top of that? 215 00:13:22,520 --> 00:13:25,120 Is it mostly like when it's requested and you don't have 216 00:13:25,120 --> 00:13:29,160 that device, you just add it to your arsenal? 217 00:13:30,080 --> 00:13:32,240 Yes. So we have a procurement and 218 00:13:32,240 --> 00:13:38,040 operations team and they're responsible for procuring the 219 00:13:38,040 --> 00:13:40,840 devices, but also all the hardware that we kind of connect 220 00:13:40,840 --> 00:13:43,880 up to the device. So a big part of what we do is 221 00:13:43,880 --> 00:13:45,400 the device management part of it. 222 00:13:45,400 --> 00:13:51,240 It's incredibly tricky to keep hundreds, thousands of TV's, you 223 00:13:51,240 --> 00:13:55,400 know, Roku's, Fire TV's, Apple TV's, iPhones, whatever, 224 00:13:55,440 --> 00:13:57,120 whatever, name it, you know, we have like. 225 00:13:57,200 --> 00:14:00,960 All the different models and like how, how many years back do 226 00:14:01,000 --> 00:14:03,960 you go like? Oh, yeah, it's, it's a massive 227 00:14:03,960 --> 00:14:06,360 problem. And so it's the, the procurement 228 00:14:06,360 --> 00:14:09,600 challenge is one piece. It's I think the bigger 229 00:14:09,600 --> 00:14:12,520 challenge is really keeping those devices online and 230 00:14:12,520 --> 00:14:14,640 healthy. So we have. 231 00:14:15,760 --> 00:14:18,040 What's that? Someone walked past the dog. 232 00:14:18,040 --> 00:14:20,960 Yeah, yeah, my dog. Hey, shy boy. 233 00:14:20,960 --> 00:14:23,880 Get over here, buddy. Yeah, we, we got a guard dog 234 00:14:23,880 --> 00:14:26,600 here on the on the podcast and and a sleeping dog. 235 00:14:27,440 --> 00:14:29,200 She can get activated really quickly though. 236 00:14:30,960 --> 00:14:33,920 Where, where were we? We were talking about how we how 237 00:14:33,920 --> 00:14:39,320 we manage all those devices. In short, there is a lot of both 238 00:14:39,320 --> 00:14:42,920 hardware and software solutions that we use to, to manage the 239 00:14:42,920 --> 00:14:46,320 devices themselves. And that's been kind of the 240 00:14:46,320 --> 00:14:49,200 hardest part that we've been iterating at for the last three 241 00:14:49,200 --> 00:14:55,720 years is how do you uniformly manage these devices so that 242 00:14:55,720 --> 00:14:59,520 when a user starts a session with them, they kind of warm up 243 00:14:59,520 --> 00:15:03,920 and, you know, load onto the home screen and then is also 244 00:15:03,920 --> 00:15:06,000 available for like, side loading, you know, your 245 00:15:06,000 --> 00:15:07,840 application, you know, because you're testing. 246 00:15:08,040 --> 00:15:11,600 These are people who build, you know, like Netflix as an 247 00:15:11,600 --> 00:15:14,760 example, would want to, you know, take a build of the 248 00:15:14,760 --> 00:15:17,840 Netflix app and load it onto a Samsung TV. 249 00:15:19,160 --> 00:15:23,000 The the TV has to be in, you know, a special developer mode 250 00:15:23,000 --> 00:15:24,880 and ports open to be able to do that. 251 00:15:25,120 --> 00:15:27,720 That's the tricky part about what we do, is all of that 252 00:15:27,720 --> 00:15:30,520 management behind the scenes so that it's a seamless experience 253 00:15:30,520 --> 00:15:32,240 for the end user to just test on. 254 00:15:32,240 --> 00:15:33,680 A. And then do you have to worry 255 00:15:33,680 --> 00:15:38,000 about security vulnerabilities on these devices and like bad 256 00:15:38,000 --> 00:15:42,080 actors using your platform? Oh, of course we, well, OK, it's 257 00:15:42,080 --> 00:15:45,000 a mix of things. So of course we have to do that. 258 00:15:45,400 --> 00:15:49,000 We have to, you know, mitigate all different types of threats. 259 00:15:49,840 --> 00:15:55,440 But I think the bigger thing is just keeping the, the devices 260 00:15:55,440 --> 00:15:59,680 locked down so that, you know, there's only minimal exposure 261 00:15:59,680 --> 00:16:02,760 for, you know, anyone threat is kind of isolated to, to the 262 00:16:02,760 --> 00:16:05,560 device itself and not to a larger network. 263 00:16:05,560 --> 00:16:09,840 But yeah, it's, it's a massive problem of, you know, how, how 264 00:16:09,840 --> 00:16:12,400 do you manage? It's not just like one type of 265 00:16:12,400 --> 00:16:15,240 device, right? It's many, many, many different 266 00:16:15,240 --> 00:16:17,440 types of platforms. And they all have different 267 00:16:18,120 --> 00:16:21,480 quirks and different challenges even across different years for 268 00:16:21,480 --> 00:16:23,560 the same model could be drastically different. 269 00:16:24,360 --> 00:16:26,800 So yeah, that that's a, it's a huge problem of managing that. 270 00:16:27,480 --> 00:16:31,320 Yeah, that just sounds insane, like the just the inventory of 271 00:16:31,320 --> 00:16:37,960 devices, like is it just smart, is it just TV's or do you have 272 00:16:37,960 --> 00:16:41,680 other devices too? No, we have phones, tablets, 273 00:16:42,440 --> 00:16:46,080 browsers. Yeah, we we've kind of got a mix 274 00:16:46,080 --> 00:16:50,640 of everything. And in your downtime, do you 275 00:16:50,640 --> 00:16:57,920 accept payments for bot farming? Won't answer that one. 276 00:17:00,560 --> 00:17:04,920 Yeah, it it is. It is definitely, yeah, there's 277 00:17:04,920 --> 00:17:07,520 definitely a lot of bad things that you could, you could do to 278 00:17:07,520 --> 00:17:09,200 our platform. So the thing I wanted to say 279 00:17:09,200 --> 00:17:12,400 before is that we, we, we typically don't let you know 280 00:17:12,400 --> 00:17:14,920 just anyone off off the Internet, you know, just come on 281 00:17:14,920 --> 00:17:16,760 our platform, access our devices. 282 00:17:17,240 --> 00:17:21,200 We've typically only work with enterprise clients, you know, 283 00:17:21,200 --> 00:17:25,839 who have an MSA with us and that kind of limits our exposure a 284 00:17:25,839 --> 00:17:28,280 bit so that, you know, we're only working with, with known 285 00:17:28,280 --> 00:17:31,680 entities versus kind of any random person who could just 286 00:17:31,680 --> 00:17:34,920 sign up and, you know, access with, you know, our devices. 287 00:17:36,480 --> 00:17:39,600 It it doesn't eliminate security threats, but it does kind of 288 00:17:39,600 --> 00:17:44,040 change like the type of threat that we're trying to mitigate. 289 00:17:44,800 --> 00:17:48,040 Yeah, that makes sense. Yeah, you you have less of a bad 290 00:17:48,040 --> 00:17:51,600 actor problem because they're all vetted users essentially, 291 00:17:51,600 --> 00:17:54,280 right? Unless you have a disgruntled 292 00:17:54,520 --> 00:17:56,240 employee at one of these companies. 293 00:17:56,880 --> 00:18:00,240 Well, I think, listen, I'm no security expert, but I know that 294 00:18:00,240 --> 00:18:06,280 a big threat model is internal. You know, internal users can be 295 00:18:06,280 --> 00:18:09,160 leaking information, stealing information from from any 296 00:18:09,160 --> 00:18:11,680 business. And so you kind of always have 297 00:18:11,680 --> 00:18:15,280 to think about, you know, principle of least privilege, 298 00:18:15,280 --> 00:18:18,360 role based access, all those kind of security best practices 299 00:18:18,360 --> 00:18:21,360 all apply here. Yeah, I've just been experienced 300 00:18:21,360 --> 00:18:25,080 that I just launched a new SAS product and I've been just 301 00:18:25,080 --> 00:18:27,040 getting hammered with pen testers. 302 00:18:27,480 --> 00:18:29,560 Oh man. Like I'm appreciative of the 303 00:18:29,560 --> 00:18:32,080 good fines, but most of it is just noise. 304 00:18:33,120 --> 00:18:36,400 Yeah, they'll be like, oh, you have a security vulnerability. 305 00:18:36,400 --> 00:18:38,840 If you pay me $100, I'll tell you what it is you get. 306 00:18:38,840 --> 00:18:43,080 Those right? Oh yeah, I actually like 'cause 307 00:18:43,760 --> 00:18:46,240 I don't know if you've seen any of my posts lately, but it's 308 00:18:46,800 --> 00:18:50,000 it's basically a zero knowledge like. 309 00:18:50,560 --> 00:18:51,760 Dead man switch. Right. 310 00:18:51,760 --> 00:18:54,280 Yeah, it's a kill. It's called kill switch, but 311 00:18:54,280 --> 00:18:56,920 it's zero knowledge. Zero knowledge. 312 00:18:56,960 --> 00:19:01,560 You know, document storage with dead man switches, right? 313 00:19:02,000 --> 00:19:06,800 So I, I don't, I'm not ready or I don't have a good way to like 314 00:19:06,800 --> 00:19:10,400 open source it right now, but I was like, well, how do I make 315 00:19:10,400 --> 00:19:12,600 people trust me that I'm doing everything right? 316 00:19:12,600 --> 00:19:15,080 So I like started a security bounty program. 317 00:19:15,280 --> 00:19:18,120 I was like, that's how I do it. Like I let everyone know that 318 00:19:18,280 --> 00:19:21,520 I'm like willing to pay for vulnerabilities and then you 319 00:19:21,520 --> 00:19:24,680 know, you can, you can get your name put on my Hall of Fame for 320 00:19:24,680 --> 00:19:27,680 finding stuff, but. That's a cool idea. 321 00:19:28,400 --> 00:19:30,400 Yeah. And because I was struggling, 322 00:19:30,400 --> 00:19:33,040 because I was like, like, I built this because it's 323 00:19:33,040 --> 00:19:35,560 something I want. But you also have to trust a 324 00:19:35,560 --> 00:19:38,960 small startup that they're handling your data correctly. 325 00:19:38,960 --> 00:19:41,800 Because how many nightmare stories have we heard, 326 00:19:41,800 --> 00:19:46,320 especially with vibe coding these days where people just 327 00:19:46,320 --> 00:19:48,680 have huge security vulnerabilities? 328 00:19:49,680 --> 00:19:52,080 Yeah. Have you explored any of the 329 00:19:52,080 --> 00:19:54,880 security certifications you know get your rubber stamp? 330 00:19:55,320 --> 00:19:57,200 No, I should probably start doing that. 331 00:19:57,200 --> 00:19:59,640 That's probably my next step for sure. 332 00:19:59,800 --> 00:20:01,480 I have. I have a lot to learn there. 333 00:20:02,200 --> 00:20:04,600 Open up your wallet is all I have to say I. 334 00:20:04,680 --> 00:20:07,800 Know that's the problem, right? Like any of those, any of the 335 00:20:07,800 --> 00:20:09,920 compliance stuff, you're just paying for it. 336 00:20:09,920 --> 00:20:12,760 It's. Very expensive, but you got to 337 00:20:12,760 --> 00:20:16,400 do it. I know I had to take on 338 00:20:16,400 --> 00:20:21,080 investors just to get my certifications, you know? 339 00:20:22,000 --> 00:20:24,680 Yeah, it's, it's a shame that there's such a high barrier to 340 00:20:24,680 --> 00:20:26,600 entry. You know, it'd be really 341 00:20:26,600 --> 00:20:31,640 interesting if there was more accessible, more accessible 342 00:20:31,640 --> 00:20:32,960 tools. Like there's just, there's an 343 00:20:32,960 --> 00:20:36,800 industry built around these certifications and pen testing 344 00:20:36,800 --> 00:20:38,640 and. They're a lot pay to play, you 345 00:20:38,640 --> 00:20:44,320 know, they kind of keep out. Yeah, I mean, I I think there's 346 00:20:44,320 --> 00:20:47,640 like a lot of good stuff there, but it's, you know, there's also 347 00:20:47,640 --> 00:20:50,160 a lot of bad stuff. And it's like, you know, some of 348 00:20:50,160 --> 00:20:53,160 these certifications are really just like companies going 349 00:20:53,160 --> 00:20:55,800 through the motions and not actually doing the diligence of 350 00:20:55,800 --> 00:20:57,720 being like, OK, there's a pen test. 351 00:20:57,720 --> 00:21:00,600 Like let me think deeply about, you know, the different threat 352 00:21:00,600 --> 00:21:04,240 models and no disaster recovery scenarios. 353 00:21:05,280 --> 00:21:08,920 You know, it it that that industry is kind of just, you 354 00:21:08,920 --> 00:21:12,680 know, this thing, it's like a cycle of money and I don't think 355 00:21:12,680 --> 00:21:14,640 that it's always turning out good results. 356 00:21:14,640 --> 00:21:19,720 So it'd be interesting to see kind of a more democratized, you 357 00:21:19,720 --> 00:21:23,840 know, standard that that was more accessible for, you know, 358 00:21:24,280 --> 00:21:28,320 smaller businesses to to engage with, whereas, you know, some of 359 00:21:28,320 --> 00:21:31,120 these other standards are are really expensive to get 360 00:21:31,120 --> 00:21:34,920 compliant and very time. Well, and then like I know any 361 00:21:34,920 --> 00:21:37,840 like military contracts or government contracts, you have 362 00:21:37,840 --> 00:21:40,800 to have these approvals to even be considered. 363 00:21:41,480 --> 00:21:42,040 Absolutely. Well. 364 00:21:43,760 --> 00:21:47,040 I maybe you can't say we don't have to talk about it, but do 365 00:21:47,040 --> 00:21:49,440 you have any like government contracts where you're testing 366 00:21:49,440 --> 00:21:55,080 devices or are you mostly just? Our our typical customers are 367 00:21:55,080 --> 00:21:57,960 media companies. So unless there there is 368 00:21:57,960 --> 00:22:00,920 government that's infiltrated, you know, some of these 369 00:22:01,320 --> 00:22:06,280 streaming media apps, I won't name any names, but unless 370 00:22:06,320 --> 00:22:08,560 that's that's happening behind the scenes that no. 371 00:22:09,040 --> 00:22:10,960 Is there a competition out there for you? 372 00:22:10,960 --> 00:22:15,240 Are you kind of the only one in the game with as far as like 373 00:22:15,240 --> 00:22:17,480 what you're, the way you're doing it, like hosting the 374 00:22:17,480 --> 00:22:21,360 devices and. Yeah, there's, there's 375 00:22:21,360 --> 00:22:23,320 definitely some other players in the game. 376 00:22:23,680 --> 00:22:27,920 The consistent feedback that we've gotten from our customers 377 00:22:27,920 --> 00:22:31,360 is that they've just had really frustrating experiences across 378 00:22:31,360 --> 00:22:33,760 the board. And I think mainly just because 379 00:22:33,760 --> 00:22:37,600 it's a, it's a hard problem to solve and a lot of companies go 380 00:22:37,600 --> 00:22:43,320 for scale to early or private equity backed and they just kind 381 00:22:43,320 --> 00:22:46,960 of go for the, the revenue numbers and the volume while 382 00:22:46,960 --> 00:22:48,840 sacrificing quality of the product. 383 00:22:48,840 --> 00:22:54,440 And so we've taken a very focused approach on, you know, 384 00:22:54,440 --> 00:22:57,600 having really high quality experiences and devices. 385 00:22:58,240 --> 00:23:00,880 And so that that's helped us be competitive in in the space, 386 00:23:00,880 --> 00:23:05,080 which is it's a, it's a tiny space relatively, but it's, you 387 00:23:05,080 --> 00:23:07,920 know, big opportunity with, you know, kind of all of the media 388 00:23:07,920 --> 00:23:08,760 that's out there. Yeah. 389 00:23:08,800 --> 00:23:11,040 That's cool. I feel like the barrier to entry 390 00:23:11,040 --> 00:23:16,080 just because like you, you have such a large inventory of TV's 391 00:23:16,080 --> 00:23:18,800 and devices that makes it difficult. 392 00:23:18,800 --> 00:23:22,880 Like what does that like how did you even start, you know, at the 393 00:23:22,880 --> 00:23:26,240 beginning, did you just limit the devices available for 394 00:23:26,240 --> 00:23:30,080 testing or was there like kind of strategy to to kind of device 395 00:23:30,080 --> 00:23:33,720 overlap or I don't know, what did you do there? 396 00:23:34,480 --> 00:23:38,240 Very good question. We started out with a single 397 00:23:38,240 --> 00:23:44,360 device that was in a tent. So we had, I think it was an 398 00:23:44,360 --> 00:23:50,960 LGTVI, think it was like a 2021 LG or 2022 LG. 399 00:23:51,360 --> 00:23:55,040 We had a camera pointed at it and then a tent around it. 400 00:23:55,960 --> 00:23:57,640 And that was our first prototype. 401 00:23:57,640 --> 00:24:02,440 It sat next to us in the office that we're working out of and we 402 00:24:02,440 --> 00:24:05,760 kind of just focused on getting that device like up and running 403 00:24:05,760 --> 00:24:09,400 and working, which was a tremendous effort that evolved a 404 00:24:09,400 --> 00:24:13,400 ton of, you know, computer vision and, you know, on the 405 00:24:13,400 --> 00:24:17,240 hardware side of things, you know, omitting IR, you know, 406 00:24:18,160 --> 00:24:20,800 we're, we're connecting, you know, trying to integrate with 407 00:24:20,800 --> 00:24:23,480 the device over multiple integration points. 408 00:24:23,480 --> 00:24:28,960 You have IR, you have CC, you have all the different SDKS for 409 00:24:28,960 --> 00:24:31,320 all these different platforms that are communicating over 410 00:24:32,240 --> 00:24:35,120 Internet Protocol. And so we're, we're just kind of 411 00:24:35,120 --> 00:24:37,800 like locked in and focused on getting that LG working. 412 00:24:38,320 --> 00:24:41,880 And then as we started to talk with customers, we kind of 413 00:24:41,880 --> 00:24:44,360 started to get a sense for like these are the platforms that we 414 00:24:44,360 --> 00:24:47,760 wanted to, to tackle next. So, you know, we, we just really 415 00:24:47,760 --> 00:24:50,680 focused on the US market to start and kind of the biggest 416 00:24:50,680 --> 00:24:52,680 names there. So you've got your, your 417 00:24:52,680 --> 00:24:57,640 Samsung's, your LG's, your Roku's, your TV, Apple TV's, you 418 00:24:57,640 --> 00:25:01,640 know, all those kinds of devices was, was our starting point. 419 00:25:01,880 --> 00:25:04,800 And then as we brought on new clients, you know, you get 420 00:25:04,800 --> 00:25:08,720 random requests for different specific European devices. 421 00:25:08,720 --> 00:25:12,600 Like we have a Vita TV as an example, which is, you know, not 422 00:25:12,600 --> 00:25:13,760 something you typically see in the US. 423 00:25:13,760 --> 00:25:16,120 We have the client who's like, we're trying to test on this 424 00:25:16,120 --> 00:25:18,720 particular TV and we can't get it anywhere. 425 00:25:18,720 --> 00:25:21,280 And it's really hard to test on. And so we built a bunch of 426 00:25:21,280 --> 00:25:24,160 custom tools for them to to ship logs. 427 00:25:24,400 --> 00:25:26,840 You know, actually this is really interesting. 428 00:25:26,840 --> 00:25:30,680 One of these TV's like, I guess they have browsers running 429 00:25:30,680 --> 00:25:32,440 inside of them. You know, a lot of their 430 00:25:32,440 --> 00:25:36,360 operating systems are just embedded browsers, Firefox or or 431 00:25:36,640 --> 00:25:39,720 Chrome. And some of these are just 432 00:25:39,720 --> 00:25:42,440 frozen in time. So on these Vita T VS, I think 433 00:25:42,440 --> 00:25:46,040 they're running a version of Spider Monkey or Firefox that 434 00:25:46,080 --> 00:25:50,720 only supported like part of ES6. I'm talking about JavaScript 435 00:25:50,720 --> 00:25:54,240 here, ES6, mostly ES-5. And what was happening was 436 00:25:54,240 --> 00:25:57,440 they're running their app on this TV and it was failing to 437 00:25:57,440 --> 00:25:59,960 parse, but there's no way to get logs out, so they didn't know 438 00:25:59,960 --> 00:26:04,400 why it was failing to parse. Oh man, dude, when you start 439 00:26:04,400 --> 00:26:07,880 working with hardware it just adds a layer of complexity. 440 00:26:07,960 --> 00:26:11,040 It's it gets so hard. Oh man, yeah. 441 00:26:11,040 --> 00:26:13,440 I mean talking about these like we're. 442 00:26:13,440 --> 00:26:17,240 And when we started doing hardware development, we had a 443 00:26:17,240 --> 00:26:21,280 challenge of just how do we iterate on our hardware? 444 00:26:21,520 --> 00:26:23,880 You know, there's a lot of you know, we learned a lot about the 445 00:26:23,880 --> 00:26:25,720 process. You know, we, we came at this 446 00:26:26,440 --> 00:26:30,080 very new Thomas, my other Co founder. 447 00:26:30,720 --> 00:26:35,120 He's a former Audi engineer and he's German. 448 00:26:35,120 --> 00:26:36,840 And so he. I feel like you got some really 449 00:26:36,840 --> 00:26:41,080 smart guys on your team. We have we're, we're blessed 450 00:26:41,080 --> 00:26:43,920 with an amazing team, really, really talented team across the 451 00:26:43,920 --> 00:26:48,440 board. But yeah, when, when Thomas 452 00:26:48,440 --> 00:26:50,440 started working on our hardware, we're, we're just kind of 453 00:26:50,440 --> 00:26:55,000 learning like the the basics in a lot of ways of hardware 454 00:26:55,000 --> 00:26:58,040 development iterations, like how do you, how do you build one of 455 00:26:58,040 --> 00:27:01,880 these things and do it well? And I think we, we did a lot of 456 00:27:01,880 --> 00:27:04,560 things the hard way and learned a lot of lessons the hard way 457 00:27:04,720 --> 00:27:07,160 about hardware development and software development and product 458 00:27:07,160 --> 00:27:08,800 development and sales and everything else. 459 00:27:08,800 --> 00:27:14,520 But it's that's why I say that TV Labs is such like a wide and 460 00:27:14,520 --> 00:27:18,040 deep problem because you know, on a typical day I would, I 461 00:27:18,080 --> 00:27:21,560 would joke like in the early days, I'd be working on like a 462 00:27:21,560 --> 00:27:27,000 front end CSS problem and then like turn my head and talk to to 463 00:27:27,000 --> 00:27:30,200 Thomas, my Co founder and then start working on firmware like 464 00:27:30,200 --> 00:27:33,040 in the matter of minutes. So I I called ourselves deep 465 00:27:33,040 --> 00:27:35,920 stack developers because like this is way beyond full stack. 466 00:27:36,240 --> 00:27:41,560 Yeah, for real. I in another life about 15 years 467 00:27:41,560 --> 00:27:48,120 ago now, No, yeah, 15 to 1010, like through that window I did 468 00:27:48,840 --> 00:27:52,360 like manufacturing automation and like I would tear apart old 469 00:27:52,360 --> 00:27:56,600 hydraulic machinery and then update with DLCS and like linear 470 00:27:56,600 --> 00:27:59,960 drives and things. And yeah, it's a, it's a 471 00:27:59,960 --> 00:28:04,480 definitely a different animal. It's, it's cool, it's really 472 00:28:04,480 --> 00:28:08,440 awesome when you see kind of like your software, like 473 00:28:08,440 --> 00:28:10,440 operating on things in the physical world. 474 00:28:10,440 --> 00:28:14,280 It's a starting. Move like to build a product 475 00:28:14,280 --> 00:28:16,760 with software you built like it's it is cool. 476 00:28:18,520 --> 00:28:21,640 It like I, I always kind of miss it where I kind of want to start 477 00:28:21,640 --> 00:28:25,320 just like an IoT lab, a little desk over here to the side so I 478 00:28:25,320 --> 00:28:29,120 can just like tinker with some, some embedded hard hardware 479 00:28:29,120 --> 00:28:31,480 stuff. Yeah, it's fun. 480 00:28:31,600 --> 00:28:34,760 It's really cool. We have all different things, 481 00:28:34,760 --> 00:28:37,360 the hardware that we played with and worked on. 482 00:28:38,160 --> 00:28:41,040 I'm sure, yeah, that's, that's crazy. 483 00:28:41,040 --> 00:28:43,800 I, yeah, you're, you're doing a lot. 484 00:28:44,080 --> 00:28:47,880 I'm always impressed. Like 'cause like building a SAS 485 00:28:47,880 --> 00:28:51,240 product or, you know, just a software product in general is 486 00:28:51,240 --> 00:28:55,240 relatively easy, right? And then just once you have to 487 00:28:55,240 --> 00:28:59,680 start inventorying physical things and you just add so much 488 00:28:59,680 --> 00:29:02,920 complexity, like you have to have a seamless system for 489 00:29:02,920 --> 00:29:07,480 people to test on physical devices remotely, and that's 490 00:29:07,480 --> 00:29:10,600 impressive. Thank you. 491 00:29:10,600 --> 00:29:12,840 It's it's definitely a big challenge and I wouldn't say 492 00:29:12,840 --> 00:29:14,880 that just software is easy by the way, but. 493 00:29:15,520 --> 00:29:17,360 Well. It's definitely an extra 494 00:29:17,360 --> 00:29:20,560 challenge of hardware. Because like I joke too, like, 495 00:29:20,560 --> 00:29:23,760 because I'm just like I'm a very technical, I like to build 496 00:29:23,760 --> 00:29:28,480 things, solve problems and I call like building the platform, 497 00:29:28,480 --> 00:29:31,440 the easy part. And then then now you have to 498 00:29:31,440 --> 00:29:33,440 start selling and actually get people to want to use it. 499 00:29:33,440 --> 00:29:35,360 That's the hard part, at least for me. 500 00:29:37,080 --> 00:29:38,320 It's all hard. It's all hard. 501 00:29:38,320 --> 00:29:39,680 It's all I have to say that's. True. 502 00:29:40,880 --> 00:29:47,160 Fair enough. Let's let's kind of talk about 503 00:29:47,160 --> 00:29:50,760 your stack and where. Where does Elixir fit in? 504 00:29:50,760 --> 00:29:54,720 How do you use Elixir and why? Why did you land on using 505 00:29:54,720 --> 00:29:57,800 Elixir? Yeah, absolutely. 506 00:29:59,800 --> 00:30:04,600 Well, I guess why Elixir for me kind of starts back at 507 00:30:04,600 --> 00:30:06,960 Bloomberg. So the beginning of of our chat 508 00:30:06,960 --> 00:30:11,960 where I was working on this big JavaScript code base for 509 00:30:11,960 --> 00:30:18,400 bloomberg.com, we built one of the first isomorphic web 510 00:30:18,400 --> 00:30:20,560 frameworks. So like when we were building 511 00:30:20,560 --> 00:30:24,600 bloomberg.com, like React happened like as we're as we're 512 00:30:24,600 --> 00:30:28,120 working on it. And one of the things that we 513 00:30:28,120 --> 00:30:33,600 did, this guy Wayne Warner, who I worked with at Bloomberg, 514 00:30:33,600 --> 00:30:38,880 who's now at NBC, he built Bloomberg Brisket, which was 515 00:30:38,880 --> 00:30:43,320 this isomorphic web framework built on BACMO and JS, if you 516 00:30:43,320 --> 00:30:44,520 remember BACMO. And. 517 00:30:45,640 --> 00:30:47,640 JS. So I'm all in on the JavaScript 518 00:30:48,360 --> 00:30:54,600 ecosystem and my manager was like, you know, I was talking 519 00:30:54,600 --> 00:30:57,080 about Elm, how I thought Elm and functional languages were cool. 520 00:30:57,080 --> 00:30:58,600 And he's like, oh, you should try Elixir. 521 00:30:59,240 --> 00:31:02,120 And that just red pilled me. I was like, oh man, this thing 522 00:31:02,120 --> 00:31:04,640 is amazing. I was built a chat, a Slack bot 523 00:31:04,640 --> 00:31:07,480 with it and all I wanted to do was Elixir. 524 00:31:07,520 --> 00:31:12,720 And funny enough, this guy, Josh, Josh Topolsky, who was one 525 00:31:12,720 --> 00:31:17,920 of the founders of The Verge and related in, in Vox Media, he was 526 00:31:17,920 --> 00:31:20,360 working at Bloomberg as like the, the head on the, the 527 00:31:20,360 --> 00:31:24,400 editorial side. And he left and started a 528 00:31:24,400 --> 00:31:28,160 company called the Outline and their CTO, Evar Vaughn was 529 00:31:28,200 --> 00:31:31,200 looking for an Elixir developer. I'm like, this is it? 530 00:31:31,200 --> 00:31:34,160 This is my call. So I left, left Bloomberg, went 531 00:31:34,200 --> 00:31:36,520 to go do Elixir at, at the Outline. 532 00:31:37,400 --> 00:31:41,520 And ever since then I've just considered Elixir my not so 533 00:31:41,520 --> 00:31:46,760 secret weapon and it built 4 businesses now off of the list. 534 00:31:47,440 --> 00:31:49,840 All right, pause that thought. I'm going to just run some 535 00:31:50,280 --> 00:31:52,280 sponsorships here and then we'll be right back. 536 00:31:52,640 --> 00:31:56,160 Hey, I'm Kimberly with Elixir mentor Jacob recently added 537 00:31:56,160 --> 00:31:59,520 recruitment services to help companies hire pre vetted Elixir 538 00:31:59,520 --> 00:32:02,480 devs. I help with company outreach, so 539 00:32:02,480 --> 00:32:05,800 if you're hiring, I may be reaching out, or you can connect 540 00:32:05,800 --> 00:32:07,960 with us anytime at elixirmentor.com. 541 00:32:09,320 --> 00:32:12,440 Coding Elixir with AI or building AI into your Elixir 542 00:32:12,440 --> 00:32:15,160 apps? The Elixir AI Collective Discord 543 00:32:15,160 --> 00:32:18,040 is your community for both. Link in the description below 544 00:32:19,760 --> 00:32:27,960 and we're back yeah, I discovered Elixir and built my a 545 00:32:28,120 --> 00:32:30,360 failed product at this point, but I learned a lot. 546 00:32:30,360 --> 00:32:35,120 I built like a, a fitness marketplace and I started 547 00:32:35,120 --> 00:32:37,760 learning Elixir struggled a little bit because it was my 548 00:32:37,760 --> 00:32:40,120 first functional immutable language. 549 00:32:41,360 --> 00:32:45,240 And then I fell in love. Like first I fell in love with 550 00:32:45,240 --> 00:32:49,160 the syntax, then I fell in love with like the ability you have 551 00:32:49,360 --> 00:32:53,080 building on the beam. And I think like that's eye 552 00:32:53,080 --> 00:32:56,880 opening, like what, what it's capable of doing and scaling it 553 00:32:56,880 --> 00:33:00,800 and it's just, it's it's cool. The Actor model can't. 554 00:33:01,840 --> 00:33:03,520 You can't get that anywhere else. 555 00:33:04,880 --> 00:33:08,480 Yeah, I mean, listen, that's kind of how I got into it too. 556 00:33:08,480 --> 00:33:12,080 Is like I got really, I got fascinated by pattern matching, 557 00:33:12,080 --> 00:33:15,840 so syntax and I'm like, wow, this is like I've seen this in 558 00:33:15,840 --> 00:33:18,280 other languages, but not in the successful way. 559 00:33:18,280 --> 00:33:19,920 And that's like what really hooked me. 560 00:33:20,360 --> 00:33:22,880 Then I, I started building chat bots and, you know, at the 561 00:33:22,880 --> 00:33:27,440 outline, we're doing things like running a process for every user 562 00:33:27,440 --> 00:33:30,640 so that we could personalize contents or kind of like just 563 00:33:30,640 --> 00:33:32,800 streaming whatever. They were content, they were 564 00:33:32,800 --> 00:33:35,360 viewing, we were streaming and they were like doing basic 565 00:33:35,360 --> 00:33:38,080 algorithms to like serve them kind of the next content. 566 00:33:38,600 --> 00:33:41,720 And that would be almost impossible to do well in like 567 00:33:41,720 --> 00:33:44,680 JavaScript or like. Oh yeah. 568 00:33:44,680 --> 00:33:47,320 Node like you would have so many problems. 569 00:33:47,880 --> 00:33:52,000 Well, you'd have multiple micro services and Redis, Rabbit, MQ 570 00:33:52,000 --> 00:33:53,880 or coffee, I don't know whatever you'd have. 571 00:33:53,880 --> 00:33:57,720 But I mean, the unlock for me and why I've stuck with Elixir 572 00:33:57,720 --> 00:34:01,160 so much is that it just puts everything in reach in a way 573 00:34:01,160 --> 00:34:07,840 that I I've worked in a lot of JavaScript in, in C++ and 574 00:34:07,840 --> 00:34:10,960 Python. You know, I've done PHP, Ruby, 575 00:34:12,480 --> 00:34:16,199 yeah, I think the last time we talked about Rus, like I, I have 576 00:34:16,199 --> 00:34:18,920 played around with many languages go we, you know, we've 577 00:34:18,920 --> 00:34:20,440 done about a bunch of good development. 578 00:34:21,000 --> 00:34:25,560 The thing that keeps me coming back to Elixir is that all the 579 00:34:25,560 --> 00:34:28,560 tools are there. Like I've, I know I, I can build 580 00:34:28,560 --> 00:34:32,080 anything in Elixir and ship it fast and make it scale and not 581 00:34:32,080 --> 00:34:36,159 have to worry about that piece. So the, the distance between 582 00:34:36,440 --> 00:34:42,080 kind of releasing a new product and like an idea is much shorter 583 00:34:42,120 --> 00:34:44,840 in, in Elixir than I think any other platform I've seen. 584 00:34:47,120 --> 00:34:48,920 And if there was a better platform, I would use it. 585 00:34:48,920 --> 00:34:51,760 And we do use plenty of other languages when that's the right 586 00:34:51,760 --> 00:34:54,760 tool. But I just find that Elixir is 587 00:34:54,920 --> 00:34:58,160 consistently just the best tool for the job. 588 00:34:58,200 --> 00:35:01,320 And and that's, you know, a personal thing too, you know, So 589 00:35:01,960 --> 00:35:04,280 one of the reasons why I think Elixir is so great is because 590 00:35:04,280 --> 00:35:09,040 I've spent actually, I think this year is my 10th year of 591 00:35:09,040 --> 00:35:10,880 using Elixir in production. You're an. 592 00:35:10,960 --> 00:35:14,520 OG dude, you're. An OG yeah, in in September of 593 00:35:14,520 --> 00:35:18,640 this year will be 10 years of not just I've I've crossed the 594 00:35:18,640 --> 00:35:21,920 10 year over 10 years of playing with Elixir, but ten years of 595 00:35:21,920 --> 00:35:25,240 elixir in production. For me, that's awesome September 596 00:35:25,320 --> 00:35:28,680 of this year. And the thing that keeps me 597 00:35:28,680 --> 00:35:31,880 coming back, you know, is that ease of, you know, access and 598 00:35:31,880 --> 00:35:35,560 just ability to get things done. But I think it's also the 599 00:35:35,560 --> 00:35:38,880 community building part of it. You know, we come back, coming 600 00:35:38,880 --> 00:35:40,960 back to TV Labs and having a great team. 601 00:35:41,880 --> 00:35:45,880 What's been just so amazing about my journey in the Elixir 602 00:35:45,880 --> 00:35:50,520 community is just how accessible everyone is. 603 00:35:50,720 --> 00:35:55,080 I mean, you know, talking to someone like Jose or or Chris 604 00:35:55,080 --> 00:35:58,960 McCord is really only a chat away and they're very engaged in 605 00:35:58,960 --> 00:36:02,000 the community. And, you know, just so many 606 00:36:02,000 --> 00:36:05,480 friends around the world that I've met, you know, as far as, 607 00:36:05,480 --> 00:36:11,760 you know, my buddy Marcelo down in, in Uruguay, or, you know, 608 00:36:11,760 --> 00:36:15,720 friends that I have, you know, in Europe, I, I really, you 609 00:36:15,720 --> 00:36:20,040 know, I employ someone Coco who's in Tokyo, Japan. 610 00:36:20,640 --> 00:36:24,440 I've met people literally all around the globe in, in 611 00:36:24,440 --> 00:36:26,320 different hemispheres from Elixir. 612 00:36:27,000 --> 00:36:30,360 And that's just such a powerful recruiting network that I just 613 00:36:30,360 --> 00:36:32,400 can't imagine in any other community. 614 00:36:32,800 --> 00:36:35,360 Yeah, dude, the elixir is a special place. 615 00:36:35,360 --> 00:36:38,800 I like it, like it's opened a lot of doors for me and like 616 00:36:38,840 --> 00:36:40,960 here I am like I was late to the party. 617 00:36:40,960 --> 00:36:44,000 I didn't even know what Elixir was till 2021. 618 00:36:44,640 --> 00:36:47,200 And I was like, you know what, I'm starting a YouTube channel 619 00:36:47,200 --> 00:36:48,560 because I struggled a little bit. 620 00:36:48,560 --> 00:36:50,560 So I'm going to help help people. 621 00:36:52,120 --> 00:36:55,480 And then now it's migrated into a podcast as well, and I've met 622 00:36:55,480 --> 00:36:57,120 really awesome people because of it. 623 00:36:59,160 --> 00:37:03,040 It's waving to the chat. It's, it's a, it's amazing 624 00:37:03,040 --> 00:37:05,160 community. So yeah, I, I think when I 625 00:37:05,520 --> 00:37:08,760 coming back to your question around, you know, why, why 626 00:37:08,760 --> 00:37:12,920 Elixir and how do we use it? The, the why is really, you 627 00:37:12,920 --> 00:37:16,880 know, I've spent 10 years building product and kind of 628 00:37:16,880 --> 00:37:19,000 really understanding its capabilities. 629 00:37:19,000 --> 00:37:22,400 You know, Elixir, the BEAM, you know, I've gotten involved in 630 00:37:23,080 --> 00:37:25,480 Erlang pod projects like Lou Earle. 631 00:37:25,800 --> 00:37:29,160 You know, I've tested the limits of the BEAM with writing machine 632 00:37:29,160 --> 00:37:33,320 learning algorithms and, and Rust done a lot of things that 633 00:37:33,320 --> 00:37:36,440 you shouldn't do. But I've also done a lot of, 634 00:37:37,560 --> 00:37:41,000 I've done a lot of, you know, things that were just so 635 00:37:41,000 --> 00:37:44,280 radically simple because of, you know, how accessible they were 636 00:37:44,280 --> 00:37:47,160 in the beam. And yeah, that that's the, I 637 00:37:47,160 --> 00:37:51,520 think the superpower. So should I answer kind of a how 638 00:37:51,520 --> 00:37:54,000 we use Elixir now? Yeah, yeah, yeah. 639 00:37:54,360 --> 00:37:55,760 I think everyone's going to agree. 640 00:37:55,760 --> 00:37:58,160 If they write anything in Elixir, especially in 641 00:37:58,160 --> 00:38:03,200 production, it is a superpower. Before we talk about how you use 642 00:38:03,200 --> 00:38:06,200 it, I'm just kind of curious what what is the size of your 643 00:38:06,200 --> 00:38:08,920 team? Good question. 644 00:38:08,920 --> 00:38:17,960 So we've got I guess 6 full time engineers, myself included, on 645 00:38:17,960 --> 00:38:20,800 the teams. OK, So that's wild to me, right? 646 00:38:20,800 --> 00:38:24,800 Like how many startups do you look at or companies? 647 00:38:25,000 --> 00:38:27,360 You're not really a startup at this point because you're just, 648 00:38:27,840 --> 00:38:29,760 you're there and you're, you're doing your thing. 649 00:38:30,280 --> 00:38:31,920 But I still call it the startup, yeah. 650 00:38:32,240 --> 00:38:33,920 Are you? You still got the startup 651 00:38:33,920 --> 00:38:37,160 hustle. We're we're, believe me, we're 652 00:38:37,160 --> 00:38:42,840 hustling. But like you don't see that on a 653 00:38:42,840 --> 00:38:46,920 lot of teams where they're like writing JavaScript and micro 654 00:38:46,920 --> 00:38:51,440 services and now you have to be an AWS micro service expert and 655 00:38:51,720 --> 00:38:55,240 all these things and your team gets big very quickly. 656 00:38:55,640 --> 00:39:00,520 And it's so cool to see like full blown products that are 657 00:39:00,920 --> 00:39:04,120 doing a lot of heavy lifting with small engineering teams. 658 00:39:05,160 --> 00:39:07,840 Yeah, we're, we're small. I mean, I love working on a 659 00:39:07,840 --> 00:39:10,280 small team. I've always said that I wanted 660 00:39:10,280 --> 00:39:13,160 to keep the team small. I think we're a little bit too 661 00:39:13,160 --> 00:39:17,000 small right now. We're we're kind of, we have a 662 00:39:17,000 --> 00:39:22,640 lot of feature requests from a lot of big media companies who 663 00:39:22,640 --> 00:39:25,760 who want us to to do more, which I think is great. 664 00:39:25,760 --> 00:39:27,840 And we and we are really committed to doing. 665 00:39:28,640 --> 00:39:31,880 But yeah, I mean, just with the small team, like it's actually 666 00:39:31,880 --> 00:39:37,400 shocking how much gets done. You know, we recently in the 667 00:39:37,400 --> 00:39:43,200 past month and a half, we've had some issues, you know, just on 668 00:39:43,840 --> 00:39:45,440 integrations with different platforms. 669 00:39:45,440 --> 00:39:50,240 So we've been working on better integrations for iPhones and 670 00:39:50,240 --> 00:39:54,360 Apple TV's. And shout out to David 671 00:39:54,360 --> 00:39:59,240 Bernheisel, who's TV Labs engineer and another podcast 672 00:39:59,240 --> 00:40:03,680 host on Thinking Looks here. He built an entire library, an 673 00:40:03,680 --> 00:40:10,200 elixir for communicating with Apple devices over USB, over the 674 00:40:10,200 --> 00:40:15,960 network, over HTTP. And then like has this whole 675 00:40:15,960 --> 00:40:19,360 discovery layer and can talk to all like the low level Apple 676 00:40:19,360 --> 00:40:24,400 services and protocols and replace like 3 Python libraries 677 00:40:24,400 --> 00:40:27,160 with that. And so like there's, there's 678 00:40:27,160 --> 00:40:31,520 just a lot of really talented people on the team who could do 679 00:40:31,520 --> 00:40:36,480 really cool things, You know, that I I think are, are really 680 00:40:36,480 --> 00:40:39,960 hard to do in other languages. You know, there's stuff out 681 00:40:39,960 --> 00:40:41,440 there. I mean, the amazing thing about 682 00:40:41,440 --> 00:40:46,000 like the Python ecosystem, for example, is just the volume of 683 00:40:46,000 --> 00:40:48,840 different libraries out there. You know, I think there's memes 684 00:40:48,840 --> 00:40:52,000 about it and, and that's definitely not there. 685 00:40:52,000 --> 00:40:53,600 Same. With web frameworks, right? 686 00:40:55,240 --> 00:41:00,600 Like for sure, for sure. But if you, whenever I crossover 687 00:41:00,600 --> 00:41:04,640 into another ecosystem, you know, a lot of times that's 688 00:41:04,640 --> 00:41:09,040 Python And JavaScript, sometimes Rust, sometimes Go, but mostly 689 00:41:09,040 --> 00:41:11,960 the other languages I'm dealing with on a daily basis or 690 00:41:11,960 --> 00:41:17,720 JavaScript and Python. And there's just a different 691 00:41:17,720 --> 00:41:24,400 ethos, like the stability of JavaScript APIs as dependencies 692 00:41:24,400 --> 00:41:27,600 is atrocious. Like those communities do not 693 00:41:27,600 --> 00:41:33,080 value stability when it comes to like API stability and you know, 694 00:41:33,120 --> 00:41:36,360 maintaining, you know, you know, try to limit breaking changes 695 00:41:36,560 --> 00:41:38,880 just. Really not Elixir project with 696 00:41:38,920 --> 00:41:43,160 the number of dependencies used versus a JavaScript project like 697 00:41:43,280 --> 00:41:46,920 you're gonna, it's gonna be like thousands versus less than 50. 698 00:41:48,240 --> 00:41:50,960 Yeah, it's crazy. And just to do simple things 699 00:41:51,040 --> 00:41:54,080 too. And I, I think I really, you 700 00:41:54,080 --> 00:41:58,160 know, there's a lot of values that I, I really identify with 701 00:41:58,200 --> 00:42:02,240 in Elixir 1 is stability. But you know, also having just a 702 00:42:02,240 --> 00:42:05,400 great standard library that does so much out-of-the-box 703 00:42:05,400 --> 00:42:08,560 eliminates so many of those dependencies that you reach for 704 00:42:08,560 --> 00:42:12,360 in in JavaScript and JavaScript has like dramatically improved, 705 00:42:12,520 --> 00:42:16,200 I should say TypeScript. But you know, that ecosystem has 706 00:42:16,200 --> 00:42:19,240 dramatically improved over the years, but it's still there's 707 00:42:19,240 --> 00:42:23,840 still a values problem where a problem, in my opinion, where, 708 00:42:25,000 --> 00:42:30,000 you know, that stability is just not take into the same regard as 709 00:42:30,000 --> 00:42:31,960 it is in Elixir. And and that's something that's 710 00:42:32,280 --> 00:42:34,880 really powerful when you're building products, you know, 711 00:42:34,880 --> 00:42:38,160 having a stable foundation underneath you that we can rely 712 00:42:38,160 --> 00:42:40,080 on. And like what other language are 713 00:42:40,080 --> 00:42:42,560 you relying so much on the standard library? 714 00:42:42,800 --> 00:42:45,440 I mean, there's, there is many languages out there that you 715 00:42:45,440 --> 00:42:47,880 rely very much in the standard library, but having that 716 00:42:47,880 --> 00:42:51,120 standard library stable and just kind of like not really needing 717 00:42:51,120 --> 00:42:53,600 a lot of dependencies is is huge. 718 00:42:54,120 --> 00:42:59,200 Yeah, like and like also you can have a giant project and you 719 00:42:59,200 --> 00:43:03,520 know, a new version comes out and you could jump a couple 720 00:43:03,520 --> 00:43:05,960 version numbers, I feel like and you wouldn't have issues. 721 00:43:07,280 --> 00:43:10,440 Yeah, absolutely, absolutely. I've done it. 722 00:43:10,960 --> 00:43:13,600 Yeah, yeah. And the tooling and Elixir is 723 00:43:13,600 --> 00:43:19,200 even better like you know the Igniter project is, you know by 724 00:43:19,200 --> 00:43:21,640 Zach is. Dude, Zach has done so many cool 725 00:43:21,640 --> 00:43:23,800 things. One of my favorite ones too is 726 00:43:23,800 --> 00:43:26,400 usage rules. Like, such a simple concept, but 727 00:43:27,000 --> 00:43:28,640 such a powerful thing to use, Yeah. 728 00:43:29,160 --> 00:43:32,920 Yeah, Zach's has no shortage of great ideas. 729 00:43:34,600 --> 00:43:37,840 I, I still, I don't know if he's even a real person, honestly. 730 00:43:37,840 --> 00:43:42,000 Like he's, he's just two of the, he's so available. 731 00:43:42,000 --> 00:43:43,400 He's just always there, you know? 732 00:43:43,880 --> 00:43:46,680 He really is. He he just moved to New Jersey, 733 00:43:46,680 --> 00:43:49,240 so he's close by to me now. So I'm trying to meet up with 734 00:43:49,240 --> 00:43:51,720 him in person at some point. I didn't realize he moved. 735 00:43:51,720 --> 00:43:54,480 I knew we got a new a new role, but that's exciting. 736 00:43:55,400 --> 00:43:58,000 Yeah, sorry for revealing if that was a secret, Zach, 737 00:43:58,000 --> 00:43:58,680 Apologies. It's a. 738 00:43:58,880 --> 00:44:02,040 Secret now his, his, his. Fans in the wild, now we're 739 00:44:02,040 --> 00:44:05,200 going. To find him New Jersey little 740 00:44:05,200 --> 00:44:09,360 hotspot because collects the company that I work for is based 741 00:44:09,360 --> 00:44:11,640 out of New Jersey too that. Cool. 742 00:44:11,640 --> 00:44:14,400 Well, that's my my home state. Yeah, nice. 743 00:44:14,720 --> 00:44:20,840 All right. I've I've like I've been at the 744 00:44:20,840 --> 00:44:25,480 airport. North Liberty Airport, try to 745 00:44:25,480 --> 00:44:31,080 avoid it again, yeah. For real, That's that's I don't 746 00:44:31,080 --> 00:44:32,800 get to the East Coast very often. 747 00:44:34,160 --> 00:44:39,320 I wish I got to the east. Coast more yeah, I I've been, I 748 00:44:39,320 --> 00:44:42,320 don't have time right now or the bandwidth, but I think it'd be 749 00:44:42,320 --> 00:44:45,800 really fun. I was talking to Nathan Hassler 750 00:44:45,800 --> 00:44:47,920 this morning. But I think it'd be really fun 751 00:44:47,920 --> 00:44:53,280 to throw like a Salt Lake City Elixir conference and then or 752 00:44:53,280 --> 00:44:57,560 like Park City and so like kind of do elixir conference with 753 00:44:57,560 --> 00:45:04,320 some skiing. Well, Todd Resedeck through the, 754 00:45:04,440 --> 00:45:07,600 was it Mpex Mountain? I forget what he called it, but 755 00:45:07,920 --> 00:45:10,760 there was a salt. There was at least one, yeah. 756 00:45:11,120 --> 00:45:13,320 And I heard it went well. I heard it was really good. 757 00:45:14,160 --> 00:45:17,080 Yeah, I think they did the first nerves comp there. 758 00:45:17,400 --> 00:45:20,840 OK, very quickly. Yeah, I didn't get to, but it 759 00:45:20,840 --> 00:45:23,920 sounded fun. I think that was before, before 760 00:45:23,920 --> 00:45:27,080 I was around or before I was aware of things. 761 00:45:27,280 --> 00:45:29,560 I kind of just jumped into this blindly, you know? 762 00:45:31,000 --> 00:45:35,760 Yeah, makes sense, yeah. And then I was like, I can't 763 00:45:35,760 --> 00:45:37,560 believe people listen to me on YouTube. 764 00:45:39,600 --> 00:45:45,920 Same. All right, so back to TV Labs. 765 00:45:45,920 --> 00:45:50,560 What was what, what's, what was the hardest thing that you've 766 00:45:50,560 --> 00:45:54,680 overcome so far with building your product and where you 767 00:45:54,880 --> 00:45:56,640 almost like, we're like, we can't do this. 768 00:45:56,840 --> 00:45:58,600 We can't. We can't even move on. 769 00:45:59,960 --> 00:46:02,000 From a technical perspective or a technical? 770 00:46:02,000 --> 00:46:04,240 Perspective product too. We can talk about both. 771 00:46:04,240 --> 00:46:09,240 I feel like both are important. Yeah, What was the We've had so 772 00:46:09,240 --> 00:46:11,720 many hard problems. Honestly, I'm trying to 773 00:46:12,360 --> 00:46:14,160 struggling to think of what was the hardest problem. 774 00:46:14,440 --> 00:46:22,560 The first one that popped to mind is we run our product. 775 00:46:22,560 --> 00:46:24,560 When you're interacting with the device, it's important to be 776 00:46:24,560 --> 00:46:27,760 able to communicate with it, you know, so input to the device, 777 00:46:27,760 --> 00:46:30,880 but also output, so what's on the screen, you know, audio, 778 00:46:30,880 --> 00:46:35,080 things like that. And so in order to do this, we 779 00:46:35,080 --> 00:46:42,280 run a media pipeline in the the service that sits next to the 780 00:46:42,280 --> 00:46:44,200 device. So he said, how do we use 781 00:46:44,200 --> 00:46:46,360 Elixir? One of the ways we use Elixir is 782 00:46:46,560 --> 00:46:49,880 we have a computer that sits with every single device in our 783 00:46:49,880 --> 00:46:52,720 platform that's running what we call Sidecar. 784 00:46:53,520 --> 00:46:57,800 And sidecar, you know, is running different SDKS for all 785 00:46:57,800 --> 00:47:03,240 the different televisions it's running, you know, observability 786 00:47:03,240 --> 00:47:06,280 stuff for pushing logs, you know, wherever they need to go 787 00:47:06,280 --> 00:47:08,840 and no spans. But kind of the key 788 00:47:08,840 --> 00:47:13,600 functionality is that it's orchestrating all of the 789 00:47:13,680 --> 00:47:16,760 interaction with the device and, you know, communicating back up 790 00:47:16,760 --> 00:47:19,880 to our platform. And one of the pieces there is 791 00:47:20,160 --> 00:47:25,120 doing web RTC for when you're in like what we call manual session 792 00:47:25,200 --> 00:47:27,720 of, you know, I've got ATV, it's in my browser and I'm 793 00:47:27,720 --> 00:47:30,520 interacting with it. We're having for streaming the 794 00:47:30,520 --> 00:47:34,000 video and audio directly from that device to your browser. 795 00:47:35,240 --> 00:47:41,840 So we do that with a library called G Streamer and we write 796 00:47:41,840 --> 00:47:44,200 it in Rust. And one of the hardest 797 00:47:44,200 --> 00:47:47,360 challenges we had was just stabilizing that. 798 00:47:48,080 --> 00:47:51,760 So it kind of started out as a bash script and became a Python 799 00:47:51,760 --> 00:47:54,800 service. And then one day I was like, you 800 00:47:54,800 --> 00:47:57,280 know, I'm allowed to curse on the podcast. 801 00:47:57,280 --> 00:47:58,320 I'm gonna curse. Fuck it. 802 00:47:58,320 --> 00:48:00,600 We're doing it in Russ. Yeah. 803 00:48:00,680 --> 00:48:03,320 I, I still have us on my desk. We rewrote it in Russ. 804 00:48:06,440 --> 00:48:11,400 And the reason why we did that is we needed I, I still don't 805 00:48:11,400 --> 00:48:14,880 pretend that it's the best way to do it, but just kind of given 806 00:48:14,880 --> 00:48:17,280 where we were at at the time, we are dealing with a lot of 807 00:48:17,280 --> 00:48:19,840 different challenges of, you know, we need to interface with 808 00:48:19,840 --> 00:48:25,000 different, you know, ways of interacting with cameras and 809 00:48:25,360 --> 00:48:28,560 different, different hardware for getting device output. 810 00:48:30,880 --> 00:48:35,040 And it turns out that depending on the type of hardware you use, 811 00:48:35,040 --> 00:48:39,080 there's dramatically different, you know, drivers and firmware 812 00:48:39,080 --> 00:48:44,080 and at any point in in the chain of a software there it can fail. 813 00:48:45,080 --> 00:48:49,120 And so we needed like a really strong test bed where we can 814 00:48:49,120 --> 00:48:52,080 iterate on on this product and run it locally. 815 00:48:52,800 --> 00:48:55,320 And so stabilizing that I think was one of the, the biggest 816 00:48:55,320 --> 00:48:56,720 challenges that we've had to date. 817 00:48:57,520 --> 00:49:00,240 The other biggest challenge? Streaming. 818 00:49:00,480 --> 00:49:03,680 So I've had like. I've dipped my toes into doing 819 00:49:03,680 --> 00:49:06,080 some web RTC stuff and streaming video. 820 00:49:07,360 --> 00:49:11,480 I can never make it work well, like it's hard. 821 00:49:12,160 --> 00:49:14,760 Very hard. Buffering and like just like, I 822 00:49:14,760 --> 00:49:19,440 don't know, I think I hit my limit of smart smarts with that 823 00:49:19,440 --> 00:49:24,000 one, but I struggled. We've invested a lot of time and 824 00:49:24,000 --> 00:49:27,440 energy into improving that and I still think there is a lot of 825 00:49:27,440 --> 00:49:29,560 low hanging fruit improvements that we could do. 826 00:49:30,040 --> 00:49:32,440 But it's kind of good enough for for what we're trying to do 827 00:49:32,440 --> 00:49:35,600 today. And I think we're not hitting 828 00:49:35,600 --> 00:49:38,760 the limits of it yet, but there's a lot of things that we 829 00:49:38,760 --> 00:49:41,080 could do to dramatically improve how that works. 830 00:49:42,800 --> 00:49:47,600 The other hard problem that I think we've we've, we're still I 831 00:49:47,600 --> 00:49:51,320 think not where we want to be because it's just such a hard 832 00:49:51,320 --> 00:49:54,360 problem is consistently warming up devices. 833 00:49:55,880 --> 00:49:57,480 So just. So they're like instantly 834 00:49:57,480 --> 00:50:00,040 available, kind of. Instantly available. 835 00:50:00,040 --> 00:50:06,040 So it's just surprisingly hard problem of like getting any 836 00:50:06,040 --> 00:50:10,960 particular model of device to go from an off state to an on state 837 00:50:10,960 --> 00:50:15,400 and on the home screen and ready for like development mode. 838 00:50:15,400 --> 00:50:19,360 I was going to say like, I have like an LGTV and sometimes it 839 00:50:19,360 --> 00:50:23,240 just won't freaking turn on and I like I had to cycle the power 840 00:50:23,240 --> 00:50:28,480 to it and like, so I'm sure you run into weird things like that 841 00:50:28,480 --> 00:50:31,120 all the time. We have like 5 different ways of 842 00:50:31,120 --> 00:50:34,640 turning off a device, you know, whether it's through, you know, 843 00:50:34,640 --> 00:50:38,920 sending IR commands or literally like turning off a, a physical 844 00:50:38,960 --> 00:50:42,520 relay and turning it back on, doing that at different stages 845 00:50:42,520 --> 00:50:44,120 in our hardware. So like, you might want to 846 00:50:44,120 --> 00:50:47,240 reboot just the television, you might want to reboot the compute 847 00:50:47,240 --> 00:50:50,520 that it's next to, you might want to reboot like that entire 848 00:50:50,520 --> 00:50:54,240 system. Then there's, you know, the 849 00:50:54,240 --> 00:50:58,560 challenges of, OK, I'm going to communicate with this thing and 850 00:50:58,560 --> 00:51:01,600 I'm going to wait for it to tell me that it's on or off and it 851 00:51:01,600 --> 00:51:03,960 will just lie to you. We'll just say, yeah, I'm, I'm 852 00:51:04,080 --> 00:51:05,160 on. It's not on. 853 00:51:05,520 --> 00:51:08,480 So you, you can't take any one signal. 854 00:51:08,480 --> 00:51:11,040 And it's almost like you're, you're dealing with an 855 00:51:11,040 --> 00:51:14,320 unreliable narrator. And so you're you have to have 856 00:51:14,320 --> 00:51:16,960 like. Hallucinating the LLM like yeah 857 00:51:17,080 --> 00:51:18,720 I'm on bro. Yeah. 858 00:51:19,160 --> 00:51:23,200 So there's, I mean, we're, we're looking into exploring different 859 00:51:23,200 --> 00:51:27,960 types of algorithms that come out of reliability engineering 860 00:51:27,960 --> 00:51:32,320 for taking like a stream of signals and probabilistically 861 00:51:32,320 --> 00:51:36,400 determining, you know, is it on? Yeah, it's kind of like the 862 00:51:36,400 --> 00:51:38,120 halting problem that we're dealing with here. 863 00:51:38,480 --> 00:51:43,040 So it's like it's provably unsolvable, but you know, we're, 864 00:51:43,040 --> 00:51:45,440 we're going to try our damn best to get as close as we can. 865 00:51:45,440 --> 00:51:49,160 Yeah, I'm curious. Like I'm guessing like some 866 00:51:49,160 --> 00:51:51,960 devices are in higher demand than others and you probably 867 00:51:51,960 --> 00:51:56,200 have multiples of them like so this instant on thing. 868 00:51:56,200 --> 00:51:59,360 So obviously having a device instantly available is important 869 00:51:59,360 --> 00:52:01,240 to you and especially your customers. 870 00:52:01,920 --> 00:52:05,680 Do you, What do you do if you have like a rare device, you 871 00:52:05,680 --> 00:52:08,840 only have one of them or two of them and they're occupied? 872 00:52:08,840 --> 00:52:11,440 Do you have like a queuing system or how do you handle 873 00:52:11,440 --> 00:52:12,880 that? Yeah. 874 00:52:12,880 --> 00:52:14,160 I mean, it's coming back to Elixir. 875 00:52:14,160 --> 00:52:18,760 So when we first built the product, the first version of it 876 00:52:18,760 --> 00:52:21,640 was we just had a list of every devices and you kind of click on 877 00:52:21,640 --> 00:52:24,080 the one that you wanted and it was, you know, 1 to 1, you just 878 00:52:24,640 --> 00:52:29,560 get matched with that device. We ended up building what we 879 00:52:29,560 --> 00:52:34,000 call our demand system because it manages demand for devices 880 00:52:34,000 --> 00:52:35,800 and then matches them with real devices. 881 00:52:37,440 --> 00:52:47,080 And so this is a fully Elixir distributed matchmaking engine 882 00:52:47,120 --> 00:52:50,400 that takes queries. So we allow you to do pretty 883 00:52:50,400 --> 00:52:55,000 complex queries for a device. So you can say like give me a 884 00:52:55,880 --> 00:53:01,640 LGTV that has this range of Chrome driver versions and is in 885 00:53:01,640 --> 00:53:07,880 this range of years and also teleport it to Ireland, which we 886 00:53:07,880 --> 00:53:09,200 could talk about teleport in a second. 887 00:53:09,200 --> 00:53:18,280 And so we, we built a, our own implementation of Kibana query 888 00:53:18,280 --> 00:53:21,480 language, KQL. And so it's like a boolean 889 00:53:21,480 --> 00:53:24,240 expression where you can, you know, describe this like kind of 890 00:53:24,240 --> 00:53:27,720 complex query. And then our query engine has a 891 00:53:27,720 --> 00:53:32,480 real time database of every single device and we Shard it by 892 00:53:32,480 --> 00:53:35,280 platform. And there is a just a Gen. 893 00:53:35,280 --> 00:53:38,200 server that sits, you know, in the cluster is sharded around 894 00:53:38,200 --> 00:53:41,000 the cluster. And it's responsible for keeping 895 00:53:41,000 --> 00:53:44,120 track of devices in that platform and then matchmaking it 896 00:53:44,120 --> 00:53:47,120 with requests so that that queue is happening. 897 00:53:47,240 --> 00:53:50,600 It's backed by Postgres, but a lot of it is happening in ETS 898 00:53:50,600 --> 00:53:53,240 tables in memory. And then, you know, things move 899 00:53:53,240 --> 00:53:56,600 around or get, you know, deployments happen, you know, 900 00:53:56,600 --> 00:53:58,440 gets rehydrated from the database. 901 00:53:58,720 --> 00:54:02,440 But for the most part, it's happening in memory against, you 902 00:54:02,440 --> 00:54:07,080 know, doing in ETS lookup. And then it uses heavy users of 903 00:54:07,080 --> 00:54:10,080 Phoenix tracker for for tracking, you know, live device 904 00:54:10,080 --> 00:54:12,920 state. And then the devices themselves 905 00:54:12,920 --> 00:54:16,120 running Elixir are the source of truth for am I actually in use 906 00:54:16,120 --> 00:54:18,440 or not? So we enforce on the device 907 00:54:18,440 --> 00:54:22,760 themselves whether or not, you know, it's allowed to accept a 908 00:54:22,760 --> 00:54:25,000 session. So even if we, you know, kind of 909 00:54:25,000 --> 00:54:28,000 had a bug in our matchmaking engine and we tried to like 910 00:54:28,000 --> 00:54:30,560 start a session with a device that was already in a session, 911 00:54:30,560 --> 00:54:33,200 the device itself will just be like, Nope, sorry, in a session 912 00:54:33,560 --> 00:54:39,760 reject. That's cool, yeah, Because you 913 00:54:39,760 --> 00:54:42,440 don't think about things like that, and you could have a 914 00:54:42,440 --> 00:54:46,040 really bad user experience if all your devices are occupied, 915 00:54:46,040 --> 00:54:47,480 right? Yeah. 916 00:54:47,480 --> 00:54:49,800 So another piece of this is like, you know, business 917 00:54:49,800 --> 00:54:54,680 intelligence around variant, you know, device model variant 918 00:54:54,680 --> 00:54:59,120 utilization and you know, it's important to know, you know, how 919 00:54:59,120 --> 00:55:01,840 much, you know, contention is there for a particular device. 920 00:55:01,840 --> 00:55:04,160 So we can add additional capacity there to reduce 921 00:55:04,160 --> 00:55:06,760 contention. You know, we also have a, a 922 00:55:06,760 --> 00:55:10,000 licensing system that we built layered on top of this where, 923 00:55:10,000 --> 00:55:13,640 you know, users are paying for concurrent access to devices. 924 00:55:14,040 --> 00:55:16,680 And so they have to, you know, we have the system that checks 925 00:55:16,680 --> 00:55:20,360 out a license and then they're operating against that license. 926 00:55:20,360 --> 00:55:23,120 And if and only if they have a license, can they then even 927 00:55:23,600 --> 00:55:25,280 initiate that request for a device. 928 00:55:25,280 --> 00:55:29,280 So there's a lot of different systems moving, all running an 929 00:55:29,280 --> 00:55:33,400 Elixir. OK, So this might lead into your 930 00:55:33,400 --> 00:55:36,360 teleport question because I now I'm like really curious. 931 00:55:36,760 --> 00:55:40,280 You have a big warehouse in New York, right? 932 00:55:40,360 --> 00:55:43,800 And how do like latency has to be a problem? 933 00:55:44,040 --> 00:55:46,880 And you mentioned like a distributed system for queuing, 934 00:55:47,320 --> 00:55:51,000 but I'm guessing you don't have hardware all over the world for 935 00:55:51,360 --> 00:55:53,160 faster responses and things, do you? 936 00:55:53,800 --> 00:55:56,440 So we, we do have hardware that's located in other 937 00:55:56,440 --> 00:55:59,560 locations that are not Brooklyn. But for the most part, like our 938 00:55:59,560 --> 00:56:01,640 biggest device footprint is in Brooklyn. 939 00:56:02,080 --> 00:56:07,600 And latency is a problem. Like if you're in Australia and 940 00:56:07,600 --> 00:56:09,880 you're trying to do web RTC between US and your device, 941 00:56:10,360 --> 00:56:12,880 you'll notice the latency. It's, it's going to be, you 942 00:56:12,880 --> 00:56:14,880 know, can't be faster than the speed of light there. 943 00:56:15,600 --> 00:56:20,680 It's still a usable experience, but that's no not as important 944 00:56:20,680 --> 00:56:23,640 when you're running automated tests, which is where most of 945 00:56:23,640 --> 00:56:25,080 our clients are running their tests. 946 00:56:25,080 --> 00:56:27,920 And those are running in CI and GitHub Actions and are 947 00:56:27,920 --> 00:56:31,560 interacting with our device, you know, over HTTP or, or, you 948 00:56:31,560 --> 00:56:34,240 know, with our automation product, we have this tool where 949 00:56:34,240 --> 00:56:38,360 you can drag and drop like a, a workflow kind of it's like 950 00:56:38,360 --> 00:56:42,040 visual programming. Basically it's built for QA and 951 00:56:42,040 --> 00:56:45,240 that compiles to a Lua program that gets then executed on the 952 00:56:45,240 --> 00:56:47,400 device. So in that case there's no 953 00:56:47,400 --> 00:56:50,280 latency because the you know the execution is all happening 954 00:56:50,400 --> 00:56:53,280 local. So it it depends I guess. 955 00:56:53,800 --> 00:56:56,960 So like, obviously there's latency with manual testing and 956 00:56:56,960 --> 00:57:00,760 using the device manually, but if obviously if you're using 957 00:57:01,640 --> 00:57:03,560 automated tests, you're probably good. 958 00:57:04,720 --> 00:57:07,360 That's right. And we run servers in multiple 959 00:57:07,600 --> 00:57:10,840 different geographic locations. So you know, we run East Coast, 960 00:57:10,840 --> 00:57:16,040 West Coast London is our our configuration and that, you 961 00:57:16,040 --> 00:57:17,600 know, that mitigates some of the latency. 962 00:57:17,600 --> 00:57:19,560 But you know, if you're interacting with a device that's 963 00:57:19,560 --> 00:57:21,880 located far away from you physically, you're going to pay 964 00:57:21,880 --> 00:57:24,960 the the latency penalty. Yeah, that, that makes sense for 965 00:57:24,960 --> 00:57:27,320 sure. And that now I'm kind of 966 00:57:27,320 --> 00:57:33,160 curious, do you have a big like data footprint or not really? 967 00:57:33,160 --> 00:57:37,280 Like do you store a lot of customer data on databases or is 968 00:57:37,280 --> 00:57:40,400 it more like internal data just so you know about your devices 969 00:57:40,400 --> 00:57:45,560 and how they operate? We don't, I wouldn't call it big 970 00:57:45,560 --> 00:57:49,560 data, but we definitely have like I think like close to 400 971 00:57:49,560 --> 00:57:52,080 terabytes of recorded video content. 972 00:57:52,920 --> 00:57:54,640 OK, I. Didn't even think about that. 973 00:57:54,640 --> 00:57:58,520 You have to record a lot of video then yeah, your customers 974 00:57:58,520 --> 00:58:00,000 need to have access to that, right? 975 00:58:00,000 --> 00:58:03,440 Because it probably proves QA processes and whatever. 976 00:58:03,440 --> 00:58:08,960 Else, yeah, but every session by default is recorded as a a 977 00:58:08,960 --> 00:58:12,400 stream that that we store and allow you to to play back that 978 00:58:12,400 --> 00:58:15,200 session so that that's a lot of content there. 979 00:58:16,080 --> 00:58:18,960 And then for like the actual sessions themselves, I mean, we 980 00:58:18,960 --> 00:58:25,920 have, you know, millions of rows of, of sessions and 10s. 981 00:58:25,920 --> 00:58:28,680 I actually haven't even checked, but it might be, might be 982 00:58:28,760 --> 00:58:32,840 hundreds of millions of events, you know, in the in our Postgres 983 00:58:32,840 --> 00:58:35,640 database. But I wouldn't consider it a big 984 00:58:35,640 --> 00:58:37,760 data problem that we're dealing with here. 985 00:58:38,000 --> 00:58:42,160 I think there are, you know, ways in which we can, we can 986 00:58:42,160 --> 00:58:45,480 invent big data problems. But I think for where we're at 987 00:58:45,480 --> 00:58:47,800 with our product right now, that's kind of not kind of the 988 00:58:47,800 --> 00:58:51,960 biggest issue. How do you deal with LIKE 989 00:58:52,800 --> 00:58:56,040 privacy and like NDA is at that level? 990 00:58:56,040 --> 00:58:58,840 Because I'm sure a lot of things are tested that you know the 991 00:58:58,840 --> 00:59:02,480 public can't have knowledge of. Yeah, good question. 992 00:59:02,480 --> 00:59:07,680 So devices by default are on a shared model, but our shared 993 00:59:07,680 --> 00:59:12,120 model is kind of, you know, there's not really transparency 994 00:59:12,120 --> 00:59:15,640 into who's sharing what. And behind the scenes we have 995 00:59:15,960 --> 00:59:19,720 this concept of pools where we can take any device and put it 996 00:59:19,720 --> 00:59:25,520 into a particular pool and then we grant access to devices 997 00:59:25,520 --> 00:59:29,240 through those pools. So a lot of the times that we're 998 00:59:32,120 --> 00:59:34,600 working with clients, they might have needs where it's like, hey, 999 00:59:34,600 --> 00:59:36,160 we, we need this to be locked down. 1000 00:59:36,160 --> 00:59:39,560 We need dedicated devices. We allocate devices into pools 1001 00:59:39,560 --> 00:59:42,680 and they become private for just that client and then we release 1002 00:59:42,680 --> 00:59:44,560 them that way. So it really depends on kind of 1003 00:59:44,560 --> 00:59:47,360 the security needs. You know, another thing that we 1004 00:59:47,360 --> 00:59:52,320 do is we wipe clean the device after every session. 1005 00:59:52,320 --> 00:59:55,520 So we're tracking kind of what you're side loading onto the 1006 00:59:55,520 --> 00:59:58,360 device and you know, other settings that you might be 1007 00:59:58,360 --> 01:00:01,240 changing. And then we have a process that 1008 01:00:01,280 --> 01:00:03,680 runs after the session that cleans that all up. 1009 01:00:03,760 --> 01:00:07,280 And if it fails, we have a signaling system again. 1010 01:00:07,280 --> 01:00:08,640 It's so cool that you can do this in Elixir. 1011 01:00:08,640 --> 01:00:11,680 We just send up a signal. It's like, hey, we call it our 1012 01:00:11,680 --> 01:00:15,920 health, our health check system. It sends up a signal that says 1013 01:00:15,920 --> 01:00:19,240 hey I'm unhealthy for this reason and it automatically gets 1014 01:00:19,240 --> 01:00:20,880 taken out of rotation immediately. 1015 01:00:21,320 --> 01:00:24,280 So it's like. Pop sub of like message passing 1016 01:00:24,280 --> 01:00:25,840 how are you? How are you doing? 1017 01:00:26,000 --> 01:00:29,040 That so every device has a web socket connection up to our 1018 01:00:29,040 --> 01:00:33,840 platform and it's also and then the majority of data gets sent 1019 01:00:33,840 --> 01:00:35,440 over rabid MQ. OK. 1020 01:00:36,640 --> 01:00:40,720 But for that, that thing, when there's any sort of failure, any 1021 01:00:40,720 --> 01:00:43,160 sort of health check failure, we have, we built a framework for 1022 01:00:43,160 --> 01:00:46,320 defining health checks on the system that just run and like at 1023 01:00:46,320 --> 01:00:49,080 various points in time will kind of like poke and prod the 1024 01:00:49,080 --> 01:00:51,600 system. And if anything fails, it sends 1025 01:00:51,600 --> 01:00:53,680 up a signal either as a warning or an error. 1026 01:00:53,680 --> 01:00:58,360 Warnings just get logged. Errors actually trigger the 1027 01:00:58,360 --> 01:01:01,000 device to get removed out of rotation. 1028 01:01:01,960 --> 01:01:07,520 And so if there is a failure to remove, you know, content from 1029 01:01:07,520 --> 01:01:11,840 the device, it signals that warning that that automatically 1030 01:01:11,840 --> 01:01:14,680 creates an operations ticket that then our operations team 1031 01:01:14,680 --> 01:01:18,520 goes, looks at that device manually, you know, does 1032 01:01:18,520 --> 01:01:20,800 whatever needs to be done. And this could be, you know, 1033 01:01:20,920 --> 01:01:24,360 device failed to remove sideloaded build or it could be, 1034 01:01:25,360 --> 01:01:27,480 you know, the camera's not working. 1035 01:01:28,200 --> 01:01:31,920 And so kind of all these hardware failures and software 1036 01:01:31,920 --> 01:01:34,000 failures all have these different health checks that 1037 01:01:34,000 --> 01:01:35,640 then can take it out of rotation. 1038 01:01:35,640 --> 01:01:39,920 Do you have full time people that are just at your warehouse 1039 01:01:39,920 --> 01:01:42,720 or your your lab? Yeah, we, we do. 1040 01:01:42,720 --> 01:01:45,560 We have an operations team that's focused on that and then 1041 01:01:45,560 --> 01:01:49,280 we're also heavily. Like setting up new devices and 1042 01:01:49,360 --> 01:01:51,240 troubleshooting old ones. Yeah, wow. 1043 01:01:51,560 --> 01:01:54,800 Yeah, yeah, a lot. Very hardware focused. 1044 01:01:55,640 --> 01:02:01,960 We're also building a, a lot of tooling that is now taking it to 1045 01:02:01,960 --> 01:02:05,800 the next level and also trying to do automatic intervention to 1046 01:02:05,800 --> 01:02:10,920 bring devices back into health by like known and even unknown 1047 01:02:10,960 --> 01:02:13,320 paths. So we, we kind of dog food our 1048 01:02:13,320 --> 01:02:15,320 own products. So I, I mentioned that drag and 1049 01:02:15,320 --> 01:02:20,200 drop workflow designer, we write workflows in that and then we 1050 01:02:20,200 --> 01:02:24,280 trigger if we, the particular health check fails that then 1051 01:02:24,440 --> 01:02:27,160 the, the system we call is cable guy, like the, the Jim Carrey 1052 01:02:27,160 --> 01:02:32,200 movie, Paula Valenti, who I'm sure you know, he, he named it. 1053 01:02:33,360 --> 01:02:36,640 Oh yeah, he's great. He he's been on my podcast, 1054 01:02:36,920 --> 01:02:40,160 Smart Dude. Great, great, great guy, Valo. 1055 01:02:41,120 --> 01:02:43,200 He's working on the system called Cable Guy, which will 1056 01:02:43,200 --> 01:02:49,240 then invoke that automated tasks to run and then bring the device 1057 01:02:49,320 --> 01:02:52,920 back into health automatically. And then we're also building 1058 01:02:52,920 --> 01:02:56,800 agents that can also do different types of triage, have 1059 01:02:56,800 --> 01:02:58,920 access to different runbooks in our system. 1060 01:02:58,920 --> 01:03:02,240 So there's like a lot of really cool stuff that we're doing 1061 01:03:02,240 --> 01:03:05,520 behind the scenes to like drive up those reliability and health 1062 01:03:05,520 --> 01:03:06,360 numbers. All right. 1063 01:03:06,360 --> 01:03:11,040 I want to dive into AI, but well, first we have a question 1064 01:03:11,040 --> 01:03:13,080 here. Have you thought of connecting 1065 01:03:13,080 --> 01:03:17,880 to the CTV ads ecosystem? It's super fragmented today and 1066 01:03:17,880 --> 01:03:19,880 measurement is a mess for marketers. 1067 01:03:20,520 --> 01:03:26,360 Yes, yes, we, we do have some customers that are on the 1068 01:03:26,360 --> 01:03:32,400 platform that our ads teams. I think there is a much deeper 1069 01:03:32,400 --> 01:03:34,960 integration that we're extremely interested in. 1070 01:03:34,960 --> 01:03:39,240 But to be very honest with you, just the amount of demand for 1071 01:03:39,240 --> 01:03:42,880 just media companies wanting to to use our platform is really 1072 01:03:42,880 --> 01:03:46,040 high that we haven't, you know, even gone full into the the ads 1073 01:03:46,040 --> 01:03:47,600 ecosystem yet. But I think there's a huge 1074 01:03:47,600 --> 01:03:49,200 market there that we could be tapping into. 1075 01:03:49,200 --> 01:03:49,880 Yeah. I can't imagine. 1076 01:03:49,880 --> 01:03:52,680 I didn't even think about that, but yeah, I can imagine. 1077 01:03:53,760 --> 01:03:57,960 Yeah, it's, it's honestly, there's so many things that we'd 1078 01:03:57,960 --> 01:04:02,200 never even considered that are, you know, our platform really 1079 01:04:02,200 --> 01:04:05,480 can solve a lot of problems for a lot of companies who are 1080 01:04:05,480 --> 01:04:09,040 dealing with all sorts of problems that they require 1081 01:04:09,040 --> 01:04:12,080 running on physical devices. Yeah. 1082 01:04:12,520 --> 01:04:17,800 Dang, that's cool. Yeah, we're going to just next 1083 01:04:17,800 --> 01:04:20,480 time you'll be able to fly your jet out and we can do an in 1084 01:04:20,480 --> 01:04:24,240 person podcast, you know. A jet. 1085 01:04:24,320 --> 01:04:30,440 Yeah, yeah, sure. All right, back now, I want to 1086 01:04:30,440 --> 01:04:37,200 go down the AI because I, I know like there's a lot of AI 1087 01:04:37,200 --> 01:04:40,960 fatigue, right? But I think that like the 1088 01:04:40,960 --> 01:04:46,440 elephant in the room is we are very aware of the improvements 1089 01:04:46,440 --> 01:04:49,000 we've seen just over the last like 6 months, right? 1090 01:04:49,360 --> 01:04:52,520 Like I rarely am physically writing a lot of code anymore. 1091 01:04:52,520 --> 01:04:56,240 I do a lot of code review, but for the most part, I don't know 1092 01:04:56,240 --> 01:05:02,280 if I know how to type anymore. And it doesn't just like that 1093 01:05:02,760 --> 01:05:04,640 that will push that aside. I get it. 1094 01:05:04,720 --> 01:05:07,800 Like we have our own work flows for coding features, writing 1095 01:05:07,800 --> 01:05:11,800 code. The thing that I think is cool 1096 01:05:11,920 --> 01:05:17,880 is automating processes that aren't even building code 1097 01:05:17,880 --> 01:05:21,440 features anymore, like just day-to-day operations at 1098 01:05:21,440 --> 01:05:24,600 companies. And I'm, I'm really curious like 1099 01:05:24,880 --> 01:05:29,200 are have you implemented work flows and how are you leveraging 1100 01:05:29,360 --> 01:05:36,640 AI to help your team? I just kicked off 3 agents who 1101 01:05:36,640 --> 01:05:40,800 are going to be opening up PRS to my Lua library right now. 1102 01:05:40,800 --> 01:05:46,360 So if that answers your question, we're heavily into AI 1103 01:05:46,360 --> 01:05:48,240 agents and, and work flows there. 1104 01:05:49,960 --> 01:05:51,760 Yeah, let me tangent on something. 1105 01:05:51,760 --> 01:05:56,640 So like I said, I've been using AI since pretty much the 1106 01:05:56,640 --> 01:06:02,080 beginning of TV Labs, like a lot, a lot of the problems that 1107 01:06:02,080 --> 01:06:04,520 we've had to solve. Like I don't know how we would 1108 01:06:04,520 --> 01:06:08,400 have solved them as quickly as we did without the help of, you 1109 01:06:08,400 --> 01:06:13,360 know, ChatGPT, you know, you know, GBT 3, I'm talking about 1110 01:06:13,360 --> 01:06:19,480 like, you know, three years ago. And those tools are ancient 1111 01:06:19,480 --> 01:06:24,720 history as far as I'm concerned. And the last three months in 1112 01:06:24,720 --> 01:06:29,560 particular have felt very, very different, Like I have always 1113 01:06:29,560 --> 01:06:34,800 been, you know, deep in the code as a, you know, as CTO. 1114 01:06:35,160 --> 01:06:38,080 And it's, it's getting harder and harder for me to kind of 1115 01:06:38,080 --> 01:06:41,040 keep up with the team. They're so talented and I get 1116 01:06:41,040 --> 01:06:42,160 pulled into a lot of business stuff. 1117 01:06:42,560 --> 01:06:43,800 That's a good problem to have though. 1118 01:06:45,080 --> 01:06:46,280 Having a good it is a good problem. 1119 01:06:46,280 --> 01:06:49,680 You can't keep up with. It's a great problem to have, 1120 01:06:51,080 --> 01:06:57,800 but the last three months have really been eye opening when it 1121 01:06:57,800 --> 01:07:02,200 comes to the capabilities of and I'm I'm specifically referring 1122 01:07:02,200 --> 01:07:05,200 to the anthropic models, which I lean into the most. 1123 01:07:06,280 --> 01:07:13,800 You know, sonnet 4.5 and Opus 4.6 now are incredible tools. 1124 01:07:14,240 --> 01:07:18,680 So I've been just all, all I can think about is how can I run 1125 01:07:18,800 --> 01:07:23,080 more agents at around the clock at all points in time? 1126 01:07:23,960 --> 01:07:27,400 And, and this is not because I'm like, oh, I'm an AI maximalist 1127 01:07:27,400 --> 01:07:29,440 or anything. It's more about these are really 1128 01:07:29,440 --> 01:07:33,920 effective tools for pushing our business forward and getting the 1129 01:07:33,920 --> 01:07:38,120 whole team to use these tools to make them better, more effective 1130 01:07:38,120 --> 01:07:44,200 and impactful engineers is it's really a game changer in terms 1131 01:07:44,200 --> 01:07:47,040 of what a small team or any size team can do. 1132 01:07:47,920 --> 01:07:51,760 And so it's it's been very exciting to kind of dive into 1133 01:07:51,760 --> 01:07:56,080 that world. I feel like it just it changes 1134 01:07:56,080 --> 01:08:00,040 it like the solar, the solo, like me for instance, I can 1135 01:08:00,040 --> 01:08:03,400 build products and I not a front end dev at all. 1136 01:08:03,960 --> 01:08:07,600 Like back end is where I feel comfortable and where I excel. 1137 01:08:07,800 --> 01:08:11,800 But now I can, I can do everything myself and it's 1138 01:08:11,800 --> 01:08:13,440 pretty. It's like a superpower, you 1139 01:08:13,440 --> 01:08:15,000 know? It is a superpower. 1140 01:08:15,160 --> 01:08:19,040 It's it's terrifying actually how capable these models are. 1141 01:08:19,960 --> 01:08:23,640 But just last week alone, we were working on kind of like 1142 01:08:23,920 --> 01:08:27,680 what's called a special project for a client of ours, and we 1143 01:08:27,680 --> 01:08:33,160 needed to build this whole inventory system tracking 1144 01:08:33,160 --> 01:08:34,640 system. We had to add a bunch of 1145 01:08:34,640 --> 01:08:39,640 features to our Lua APIs to expose new functionality. 1146 01:08:39,640 --> 01:08:43,439 We had to add build these new dashboards and just a ton of 1147 01:08:43,439 --> 01:08:46,760 functionality. I wrote about 60,000 lines of 1148 01:08:46,760 --> 01:08:50,040 code last week and I wrote almost zero of them by hand. 1149 01:08:50,960 --> 01:08:55,200 They're all well tested. They all follow like idiomatic 1150 01:08:55,200 --> 01:08:59,080 best practices with Elixir. Like, no, it doesn't look like 1151 01:08:59,080 --> 01:09:01,399 code that I would have written, but it's, it's good enough. 1152 01:09:01,520 --> 01:09:06,840 And, you know, throwing property tests on different areas, It's, 1153 01:09:07,439 --> 01:09:10,240 it's absolutely crazy. You know what, what you can do 1154 01:09:10,240 --> 01:09:12,960 with these models. And, you know, one of the issues 1155 01:09:12,960 --> 01:09:16,200 that we've run into in the last few weeks that's been a repeated 1156 01:09:16,200 --> 01:09:21,439 issue for two years now is I built this library called Lua 1157 01:09:22,279 --> 01:09:27,240 and it, it sits on top of Lua, which is a Lua implement, Lua 1158 01:09:27,240 --> 01:09:30,920 5.3 implementation running entirely in the BEAM, just all 1159 01:09:30,920 --> 01:09:35,880 implemented in Erlang. And the error messages on it 1160 01:09:35,880 --> 01:09:39,560 are, are very Erlang. They're, they're terse. 1161 01:09:39,560 --> 01:09:43,439 They're, you know, in need of, you know, a little bit more 1162 01:09:43,439 --> 01:09:46,600 information. And you know, I've always wanted 1163 01:09:46,600 --> 01:09:50,560 to go and like work on Lu URL and improve the error messages. 1164 01:09:50,560 --> 01:09:57,000 But I just thought to myself, with Opus 4.6, why would I do 1165 01:09:57,000 --> 01:10:01,200 that when I could just write an implementation of Lua 5.3 1166 01:10:01,200 --> 01:10:03,520 myself? And so last Friday I just 1167 01:10:03,520 --> 01:10:07,440 started doing that. And I have, I'm this close to 1168 01:10:07,440 --> 01:10:11,920 having the full test suite of Lua 5.3 passing, which means I 1169 01:10:11,920 --> 01:10:13,560 wrote in a week. And by the way, I could have 1170 01:10:13,560 --> 01:10:15,880 written this in a single day. I've just been slowing it down 1171 01:10:15,880 --> 01:10:18,480 so that I understand every PR that's coming through. 1172 01:10:19,040 --> 01:10:22,040 But I I could have probably built this entire thing in a day 1173 01:10:22,040 --> 01:10:24,320 if I just let it Ralph to the end. 1174 01:10:25,120 --> 01:10:27,600 I'm curious like what? What does your code coverage 1175 01:10:27,600 --> 01:10:31,600 look like now with like using Anthropic to help you write code 1176 01:10:31,600 --> 01:10:34,720 and features. Like code coverage in terms of 1177 01:10:34,720 --> 01:10:36,920 like test coverage. Yeah, test coverage, yeah. 1178 01:10:38,560 --> 01:10:41,960 Well, actually, in order to do this project, there's a bunch of 1179 01:10:41,960 --> 01:10:44,760 code that we had that had really abysmal test coverage. 1180 01:10:44,760 --> 01:10:48,160 And so the first thing I did was I wrote a Claude skill and 1181 01:10:48,160 --> 01:10:52,280 taught it how to like measure test coverage and then write 1182 01:10:52,280 --> 01:10:57,920 tests and just loop until like a set of APIs all had 100% test 1183 01:10:57,920 --> 01:11:00,160 coverage that was going. To start, you've actually 1184 01:11:00,160 --> 01:11:03,440 comfortably gotten to 100%. I find it really hard to get to 1185 01:11:03,440 --> 01:11:09,120 100% test coverage that's why. I I'm being hyperbole, no, like 1186 01:11:09,480 --> 01:11:12,680 anywhere between 90 and 100. So no, it is very hard to get 1187 01:11:12,680 --> 01:11:14,160 100 and that's not even the goal. 1188 01:11:14,400 --> 01:11:17,120 The goal is just like, have confidence in changing it. 1189 01:11:17,280 --> 01:11:19,760 I'm about to RIP out the underlying library. 1190 01:11:20,480 --> 01:11:23,400 So that was my starting point was get testing up and then in 1191 01:11:23,400 --> 01:11:28,520 this, you know, new implementation of Lua, I now 1192 01:11:28,520 --> 01:11:32,280 have property tests on like all different sorts of AP is. 1193 01:11:32,680 --> 01:11:38,360 I had it write a mixed task that can go to the official Lua 1194 01:11:39,800 --> 01:11:43,680 implementation, download their test suite and then spit it out 1195 01:11:43,680 --> 01:11:47,200 in a way that I could execute from X unit and then just rerun 1196 01:11:47,200 --> 01:11:49,640 that. That's awesome. 1197 01:11:50,120 --> 01:11:52,280 Yeah. So testing is kind of like core 1198 01:11:52,280 --> 01:11:56,520 to, I think, like if you're using agents and you're not 1199 01:11:56,520 --> 01:12:00,120 giving them really strong feedback loops, you're making a 1200 01:12:00,120 --> 01:12:01,360 huge mistake. Yes. 1201 01:12:01,400 --> 01:12:05,840 And I, well, I feel like those, the people that don't like using 1202 01:12:06,240 --> 01:12:10,680 LLMS to help them, they're, they're like bad at prompting, 1203 01:12:10,680 --> 01:12:12,360 they're bad at guiding and planning. 1204 01:12:12,360 --> 01:12:15,640 I don't know the problem because like you guys are missing 1205 01:12:16,280 --> 01:12:19,520 missing out. Yeah, there's, it seems there's 1206 01:12:19,520 --> 01:12:25,840 a lot of people who either are unwilling to like jump into the 1207 01:12:25,840 --> 01:12:30,160 deep end or maybe they had an experience with LMS, you know, 1208 01:12:30,160 --> 01:12:33,440 from six months, a year, 2 years, three years ago. 1209 01:12:33,440 --> 01:12:35,000 And they're like, oh, yeah, they suck. 1210 01:12:35,000 --> 01:12:39,680 And it's like if you're doing that like, you're like, the 1211 01:12:39,680 --> 01:12:41,600 world is gonna change around you very fast. 1212 01:12:41,600 --> 01:12:44,880 And it already is. Like, I already feel so much 1213 01:12:44,880 --> 01:12:47,240 pressure to be like we need to keep up with the 1214 01:12:47,240 --> 01:12:50,680 state-of-the-art and it it's changing on a weekly basis. 1215 01:12:50,680 --> 01:12:54,520 It's hard, like when you thought being in tech was hard five 1216 01:12:54,520 --> 01:12:57,120 years ago because things changed so quick. 1217 01:12:57,520 --> 01:13:01,680 It's just accelerated now. Yeah, as far as I'm concerned, 1218 01:13:01,680 --> 01:13:03,960 it's either you get on the train or the train's just going to 1219 01:13:03,960 --> 01:13:06,440 leave the station without you. And I think I believe the 1220 01:13:06,440 --> 01:13:10,040 companies who get on that train are going to be the ones who 1221 01:13:10,040 --> 01:13:12,600 have a fighting chance of, you know, having products that 1222 01:13:12,600 --> 01:13:17,000 survive whatever the world looks like, you know, in the next many 1223 01:13:17,000 --> 01:13:18,640 years. And it's going to be different. 1224 01:13:18,640 --> 01:13:20,600 Like that's only thing I'm certain of. 1225 01:13:20,600 --> 01:13:23,080 I don't know about developer jobs or anything else. 1226 01:13:23,080 --> 01:13:25,440 I just know that the world is going to be different in the 1227 01:13:25,440 --> 01:13:28,280 future. And if you're the ones using the 1228 01:13:28,360 --> 01:13:31,440 AI tools and you deeply understand how they work and the 1229 01:13:31,560 --> 01:13:34,320 ecosystem around them, you're the ones who are going to, you 1230 01:13:34,320 --> 01:13:36,840 know, those are the people are going to be valuable and in this 1231 01:13:36,920 --> 01:13:38,720 new world. So, and that's a, that's a push 1232 01:13:38,720 --> 01:13:42,720 to the community of if you're not using these tools on a daily 1233 01:13:42,720 --> 01:13:46,880 basis, it's time. It is, yeah, I agree completely. 1234 01:13:47,840 --> 01:13:54,640 Like we're kind of turning into the orchestrator or like the the 1235 01:13:54,640 --> 01:13:57,520 architect of like a much bigger thing, right? 1236 01:13:57,520 --> 01:14:00,760 You have to understand how the pieces fit together and operate. 1237 01:14:01,240 --> 01:14:05,040 But when it comes down to just writing lines of code, we're 1238 01:14:05,440 --> 01:14:09,760 probably not doing that anymore. I yeah, I don't know. 1239 01:14:09,760 --> 01:14:12,560 I like there's always times when I'm like, I want to go back and 1240 01:14:12,560 --> 01:14:15,360 like, I want to write some code by hand and like make it feel 1241 01:14:15,360 --> 01:14:20,080 right and then have the, the stuff, the the LLM, you know, 1242 01:14:20,640 --> 01:14:24,800 base a lot of the yeah, it's like build kind of like the, the 1243 01:14:24,800 --> 01:14:27,560 bones of what you want and then have it like paint in or, you 1244 01:14:27,560 --> 01:14:30,480 know, film, have the lines and then have the LLM kind of like 1245 01:14:30,480 --> 01:14:32,880 paint in all the colors. It's really good at that. 1246 01:14:33,120 --> 01:14:37,600 What I'm starting to come around to that's very uncomfortable is 1247 01:14:38,280 --> 01:14:41,320 are you doing that because it's necessary or are you doing that 1248 01:14:41,320 --> 01:14:43,920 because it's like an ego thing and I'm I'm. 1249 01:14:45,240 --> 01:14:47,440 Yeah. Oh, I was going to say I used 1250 01:14:47,440 --> 01:14:50,040 to. I, I battled this with myself a 1251 01:14:50,040 --> 01:14:52,560 lot. I was like, I'm becoming a 1252 01:14:52,560 --> 01:14:55,760 shittier programmer because I'm writing less code. 1253 01:14:56,240 --> 01:14:58,800 But at the same time, but we're not going dark. 1254 01:14:59,800 --> 01:15:02,760 I am right, but I think I've. Got you are because there's been 1255 01:15:02,760 --> 01:15:05,840 studies on it, it's been proven that it within 6. 1256 01:15:06,120 --> 01:15:10,520 I don't even remember the time period, but it is I I don't want 1257 01:15:10,520 --> 01:15:12,200 to go ahead and say it's clinically proven, but that's 1258 01:15:12,200 --> 01:15:14,600 what it's going to say. I'll just say clinically proven 1259 01:15:17,400 --> 01:15:21,200 that that your skills do degrade by leaning on LOMS and that 1260 01:15:21,200 --> 01:15:23,720 actually gives me a ton of concern for like I have two 1261 01:15:23,720 --> 01:15:26,960 small children. Like, you know, how that impacts 1262 01:15:26,960 --> 01:15:30,000 education is, is a huge concern for me. 1263 01:15:31,040 --> 01:15:36,360 But you know, at the same time, like you, if you're not, if 1264 01:15:36,360 --> 01:15:38,680 you're doing it because you're like, I don't want my skills to 1265 01:15:38,680 --> 01:15:40,160 degrade, you're it's a trade off. 1266 01:15:40,160 --> 01:15:42,600 You're trading off it is, you know, kind of a. 1267 01:15:43,560 --> 01:15:46,680 Successful company and a really good product. 1268 01:15:46,680 --> 01:15:50,280 Then worry about my, my, my typing programming skills, you 1269 01:15:50,280 --> 01:15:52,800 know, because I still like have to solve problems. 1270 01:15:52,800 --> 01:15:54,480 We're still engineers at the end of the day. 1271 01:15:55,120 --> 01:15:57,000 You're just it's. You're shifting. 1272 01:15:57,080 --> 01:15:59,000 You're shifting how you're solving problems. 1273 01:15:59,480 --> 01:16:05,080 Your your capabilities have. 100 X. 1274 01:16:05,320 --> 01:16:07,360 Maybe even more, I don't, I don't know. 1275 01:16:07,360 --> 01:16:11,000 It's hard to quantify, but your capability like I am talking to 1276 01:16:11,000 --> 01:16:15,200 you right now and three things are getting done right now as 1277 01:16:15,200 --> 01:16:16,720 I'm speaking. Problems. 1278 01:16:16,720 --> 01:16:19,880 I'm not doing that. Why am I not writing features 1279 01:16:19,960 --> 01:16:23,280 when I podcast? Dude, why are you not doing? 1280 01:16:23,280 --> 01:16:29,600 That I'm a loser. No, but, but that's how I do 1281 01:16:29,600 --> 01:16:34,480 feel is I do feel like oh, like I'm an idiot if I don't, if I'm 1282 01:16:34,480 --> 01:16:38,120 not doing this more because it it it all comes down to what 1283 01:16:38,280 --> 01:16:40,560 what are your incentives? Are you like? 1284 01:16:40,560 --> 01:16:44,440 Are you outcome driven of I want to improve my product or you 1285 01:16:44,440 --> 01:16:47,040 know, this project, you know, am I trying to get that done or am 1286 01:16:47,040 --> 01:16:50,600 I trying to, to feel like I did it and learned something? 1287 01:16:50,600 --> 01:16:53,160 And I think there's cases where it's like, I want to write all 1288 01:16:53,160 --> 01:16:55,360 the code myself. I want to understand how this 1289 01:16:55,360 --> 01:16:56,080 works. I want to learn it. 1290 01:16:56,080 --> 01:16:57,840 And there's other times it's like, you know, I'm trying to 1291 01:16:57,840 --> 01:17:02,080 ship this feature like in an hour and it would take me 3 or 4 1292 01:17:02,080 --> 01:17:06,280 days to do this myself. And I can get this done in 20 1293 01:17:06,280 --> 01:17:11,920 minutes if I I lean on an LLM. Yeah, I'm curious, outside of 1294 01:17:11,920 --> 01:17:16,240 writing features and coding, what are you using LLMS for? 1295 01:17:17,960 --> 01:17:22,640 A big thing is just learning, Deeping it. 1296 01:17:22,800 --> 01:17:26,960 Like I can't tell you how many times I have a problem that I 1297 01:17:26,960 --> 01:17:32,240 don't understand or just kind of like a customer interaction or a 1298 01:17:32,240 --> 01:17:35,640 communication. And my first instinct now is 1299 01:17:35,640 --> 01:17:39,040 always drop it in a prompt. And I might not always use the 1300 01:17:39,040 --> 01:17:42,720 output of that directly, but it's kind of like you're 1301 01:17:42,720 --> 01:17:47,520 outsourcing a lot of that mental load to these LLMS. 1302 01:17:47,520 --> 01:17:51,320 So I think the number one thing that I do is is research and 1303 01:17:51,320 --> 01:17:55,600 kind of a communicational type stuff goes through an LLM first. 1304 01:17:56,160 --> 01:17:58,720 Another thing that we're using it is really heavily on the 1305 01:17:58,720 --> 01:18:02,440 operation side of things. So, you know, when we're trying 1306 01:18:02,440 --> 01:18:08,000 to source different components or, you know, work with a bunch 1307 01:18:08,000 --> 01:18:11,720 of different vendors, a lot of the management of that is all 1308 01:18:11,720 --> 01:18:14,800 happening in like Google Sheets. And we're using LLMS to kind of 1309 01:18:14,800 --> 01:18:18,800 like feed in information. So you can take a picture of 1310 01:18:18,800 --> 01:18:23,120 something and, you know, drop it into an LLM, ask it to pull 1311 01:18:23,120 --> 01:18:25,960 information out of that, then stick it into Google Sheets and 1312 01:18:25,960 --> 01:18:29,040 then write some Python script for you to generate some data 1313 01:18:29,040 --> 01:18:31,960 reports or whatever. Like these are things that like 1314 01:18:31,960 --> 01:18:35,360 we're training non-technical people on the team to do and 1315 01:18:35,360 --> 01:18:37,200 they don't, they're not writing a single line of code 1316 01:18:37,200 --> 01:18:38,520 themselves. It's all the LLM. 1317 01:18:38,520 --> 01:18:42,080 It's it's writing Python for them, but it's giving, you know, 1318 01:18:42,320 --> 01:18:44,800 these non-technical people on the team, these super powers to 1319 01:18:44,800 --> 01:18:48,520 do their job, you know, in a fraction of the time. 1320 01:18:48,520 --> 01:18:53,200 So that's a big thing. And then I think the biggest 1321 01:18:53,200 --> 01:18:56,400 area that we're moving into, you know, I mentioned, you know, 1322 01:18:56,400 --> 01:19:01,680 some of the LLM stuff with like, you know, operationally, like if 1323 01:19:01,680 --> 01:19:05,320 we have a customer support ticket in one of the systems 1324 01:19:05,320 --> 01:19:08,640 that we're building right now is first line of defense is that 1325 01:19:08,640 --> 01:19:12,960 gets triaged by an lol. So you know, simple thing is OK 1326 01:19:12,960 --> 01:19:16,040 user says this device is not working well. 1327 01:19:16,200 --> 01:19:19,320 The LOL then gets that, you know, running an open job, it's 1328 01:19:19,320 --> 01:19:24,120 going to pull that, that session, you know, their most 1329 01:19:24,120 --> 01:19:27,120 recent sessions, find the problem 1, you know, see if 1330 01:19:27,120 --> 01:19:30,120 there's any reported issues on that device recently, pull the 1331 01:19:30,120 --> 01:19:33,520 logs from the system and then it can all report that back for the 1332 01:19:33,520 --> 01:19:36,200 human that then is going to look at that. 1333 01:19:36,800 --> 01:19:39,440 So they have this all additional context, but the ultimate goal 1334 01:19:39,440 --> 01:19:43,520 is actually to be able to close support tickets entirely. 1335 01:19:43,680 --> 01:19:45,760 I'm curious, how are you implementing that? 1336 01:19:45,760 --> 01:19:47,560 Are you? Do you just have like an MCP 1337 01:19:47,560 --> 01:19:54,320 server running on production? And how do you trust that it 1338 01:19:54,320 --> 01:19:58,080 won't do naughty things? Good question. 1339 01:19:58,680 --> 01:20:04,560 So we have 3 or 4 different types of agents that we've 1340 01:20:04,560 --> 01:20:07,840 implemented on the system. Some are customer facing like we 1341 01:20:07,840 --> 01:20:12,040 we're currently developing an AI chat bot where you know, you 1342 01:20:12,040 --> 01:20:16,400 could be like AQA person at Warner Brothers and you're I 1343 01:20:17,680 --> 01:20:21,240 don't know, you get a customer report that a specific ties in. 1344 01:20:21,560 --> 01:20:24,000 Or Samsung device, I should say, is broken. 1345 01:20:24,240 --> 01:20:27,520 So you go into the chat and be like, hey, my app is broken on 1346 01:20:27,520 --> 01:20:29,920 this device. Can you go and navigate to this 1347 01:20:29,920 --> 01:20:33,040 part of the screen and figure out, you know, it's try to 1348 01:20:33,040 --> 01:20:36,560 reproduce the issue And our chat bot can now do that. 1349 01:20:37,000 --> 01:20:39,840 So that's cool. So we give it all these 1350 01:20:40,080 --> 01:20:45,200 different capabilities, tool calls for navigation, for taking 1351 01:20:45,200 --> 01:20:50,320 screenshots, for, you know, just pausing and waiting for an 1352 01:20:50,320 --> 01:20:52,440 action to happen for a screen to load. 1353 01:20:53,200 --> 01:20:56,920 And then actually something I really want to talk about is my, 1354 01:20:56,920 --> 01:20:59,440 my friend's project called just Bash. 1355 01:21:00,520 --> 01:21:03,920 But we give it all of these really, we give it a virtual 1356 01:21:03,920 --> 01:21:07,160 file system and allow it to kind of explore the file system and 1357 01:21:07,160 --> 01:21:10,720 just, rather than injecting stuff into the context, we give 1358 01:21:10,720 --> 01:21:15,120 it bash tools that it can then use to, to find and discover 1359 01:21:15,120 --> 01:21:17,960 information. And this is like incredibly 1360 01:21:17,960 --> 01:21:20,400 powerful. So that's one thing is these 1361 01:21:20,400 --> 01:21:23,920 agent agents that we're giving to customers. 1362 01:21:24,920 --> 01:21:29,560 We do have an MCP that we're working on for actually exposing 1363 01:21:29,560 --> 01:21:35,120 to internally so that we can use LMS to kind of do our own device 1364 01:21:35,120 --> 01:21:38,280 triage. I don't know that we will or 1365 01:21:38,280 --> 01:21:40,000 will not make that an external thing. 1366 01:21:41,880 --> 01:21:46,280 And then we have like these support and kind of like web 1367 01:21:46,280 --> 01:21:49,760 hook driven agents that it's like support ticket comes in 1368 01:21:49,760 --> 01:21:53,200 kicks off an LLM that's then going to try and like discover 1369 01:21:53,200 --> 01:21:55,800 what's what is this support ticket issue. 1370 01:21:56,000 --> 01:21:58,480 I need your. Brain about this kind of stuff 1371 01:21:58,480 --> 01:22:03,080 more because I just see that making my life so much easier 1372 01:22:03,080 --> 01:22:07,200 with support emails and we were talking before we went live 1373 01:22:07,200 --> 01:22:10,840 about pen testing emails and all that stuff. 1374 01:22:10,960 --> 01:22:12,920 It would be nice to automate that a little bit. 1375 01:22:14,000 --> 01:22:19,200 Yeah, it's it, there's so much honestly it's, it's so hard to 1376 01:22:19,200 --> 01:22:20,760 keep up with. But you know, we're, we're 1377 01:22:20,760 --> 01:22:24,120 trying to push on all different fronts honestly, just because we 1378 01:22:24,120 --> 01:22:27,280 think it's just going to be able to make such an impact to to our 1379 01:22:27,280 --> 01:22:30,280 business. Yeah, Yeah. 1380 01:22:30,280 --> 01:22:34,680 I kind of want to integrate some like local like customer facing 1381 01:22:34,680 --> 01:22:39,960 LLM stuff for kill switch, like help them fill out like notes 1382 01:22:39,960 --> 01:22:42,800 and like what kind of information should be going into 1383 01:22:42,800 --> 01:22:45,440 stuff. But then I have like the privacy 1384 01:22:45,440 --> 01:22:49,680 concern of like, does it have to be an internal LLM that doesn't 1385 01:22:49,680 --> 01:22:52,760 have access to other models? And like, I don't know, I have 1386 01:22:52,760 --> 01:22:54,760 to. I got to figure that out because 1387 01:22:54,760 --> 01:22:56,440 I don't want sensitive data leaked. 1388 01:22:57,200 --> 01:23:01,400 Yeah, I mean, listen, you can, I know there's tools out there 1389 01:23:01,400 --> 01:23:03,480 that can guarantee that you pay for them, right? 1390 01:23:03,480 --> 01:23:06,520 But you know, if you want to run a local model, I I was just saw 1391 01:23:06,520 --> 01:23:09,880 something on X today where there's like some new crazy 1392 01:23:09,880 --> 01:23:13,720 model that was released that you can run on a Mac mini. 1393 01:23:14,200 --> 01:23:19,160 So possible. Yeah, I, yeah, yeah. 1394 01:23:20,480 --> 01:23:23,840 There's just so much to learn. See, this is the problem. 1395 01:23:23,840 --> 01:23:27,200 Like we're doing less programming, but there's we're 1396 01:23:27,440 --> 01:23:29,600 1000 X in the things we have to know now. 1397 01:23:31,160 --> 01:23:32,600 There's a lot to that you need to know. 1398 01:23:32,600 --> 01:23:35,960 And I mean, again, coming back to using LMS, like I think 1399 01:23:35,960 --> 01:23:41,560 that's like my ability to explore problem spaces with LMS 1400 01:23:41,560 --> 01:23:47,600 is dramatically change. Like you know, Googling to like 1401 01:23:47,600 --> 01:23:49,920 solve a problem just seems antiquated. 1402 01:23:50,520 --> 01:23:54,040 Yeah, I agree. Yeah, times are changing. 1403 01:23:56,640 --> 01:23:58,280 They are. They really are. 1404 01:23:59,200 --> 01:24:04,880 I, I, yeah, I, I think, I don't really agree with the 1405 01:24:04,880 --> 01:24:08,120 comparison, but people are like, you know, you're going to lose 1406 01:24:08,120 --> 01:24:10,120 all your skills. You won't know how to program. 1407 01:24:10,120 --> 01:24:12,120 And then like you'll hear the people be like, well, it's just 1408 01:24:12,120 --> 01:24:13,400 a tool. It's a calculator. 1409 01:24:14,000 --> 01:24:17,400 I don't agree that it's a calculator because you get, you 1410 01:24:17,400 --> 01:24:21,520 don't get like static results every time, but it is a huge, 1411 01:24:21,520 --> 01:24:24,160 it's a tool. The only thing it does for you 1412 01:24:24,160 --> 01:24:26,800 is it makes your life better. You just have to know how to 1413 01:24:26,800 --> 01:24:30,320 properly use it because it you don't get static responses like 1414 01:24:30,640 --> 01:24:33,800 1 + 1 isn't always 2 when you're using an LLM. 1415 01:24:35,320 --> 01:24:38,560 It's true, it's true. It's, it's definitely not a, 1416 01:24:38,560 --> 01:24:42,800 it's not a calculator and it is just a tool. 1417 01:24:42,800 --> 01:24:44,800 But that's completely underselling it. 1418 01:24:44,800 --> 01:24:48,520 It's a, it's a, it's a society changing tool. 1419 01:24:48,920 --> 01:24:50,760 Yeah. Oh, agree 100%. 1420 01:24:50,920 --> 01:24:54,280 Yeah, I don't even think we understand what it's changing. 1421 01:24:55,160 --> 01:24:58,280 Yeah, it's going to change every single industry in in the world 1422 01:24:58,360 --> 01:25:03,360 and I I don't think that society is is ready or is understanding 1423 01:25:03,360 --> 01:25:05,280 of like how much is about to change. 1424 01:25:05,640 --> 01:25:09,320 Yeah, yeah, this is Yeah. I don't. 1425 01:25:09,480 --> 01:25:12,880 I think we see how cool something is, but we don't know 1426 01:25:12,880 --> 01:25:17,000 what it's going to do. It really does. 1427 01:25:17,080 --> 01:25:18,520 You can't worry about it too much. 1428 01:25:18,520 --> 01:25:21,640 I just kind of go with the flow. I wake up every day and you 1429 01:25:21,640 --> 01:25:24,160 know. Do the things you can control, 1430 01:25:24,560 --> 01:25:25,680 yeah. Exactly. 1431 01:25:25,760 --> 01:25:30,280 Like, all right, like this morning I woke up and you know, 1432 01:25:30,280 --> 01:25:34,680 I have to finish moving today and I was going to talk to Dave 1433 01:25:34,680 --> 01:25:39,040 this morning on my podcast and that that is my day. 1434 01:25:40,280 --> 01:25:42,480 Sounds like a nice day. Minus sorry you had. 1435 01:25:42,480 --> 01:25:44,480 To talk to Dave though. No, Dave. 1436 01:25:44,680 --> 01:25:48,840 Dave is pretty awesome. He's doing some cool stuff. 1437 01:25:48,880 --> 01:25:52,800 I, I think it's just so cool that talk to people that are 1438 01:25:52,800 --> 01:25:55,920 building awesome things in production and then especially 1439 01:25:55,920 --> 01:25:59,800 like you add like the IoT flavor, like the physical 1440 01:25:59,800 --> 01:26:04,480 hardware and it's just, it's so it's so much cooler to me to 1441 01:26:04,520 --> 01:26:08,840 like see all these puzzle pieces fit together to build a really 1442 01:26:08,840 --> 01:26:13,000 useful product. It's, it's, it's cool. 1443 01:26:13,040 --> 01:26:17,280 It's it's been a fun journey. TV Labs has been the most fun 1444 01:26:17,280 --> 01:26:19,200 I've ever had. I'm working on a product just 1445 01:26:19,200 --> 01:26:22,120 because there's so many big problems to dump into. 1446 01:26:22,120 --> 01:26:23,720 When did you? When did you? 1447 01:26:23,720 --> 01:26:29,880 When were you founded? 2023. 2023, OK, yeah, so not you 1448 01:26:29,880 --> 01:26:32,880 haven't been around that long. No, no. 1449 01:26:33,000 --> 01:26:36,400 But it's, it's felt like a lifetime, I'll say that. 1450 01:26:37,480 --> 01:26:39,400 Startup life always does, doesn't it? 1451 01:26:40,120 --> 01:26:42,640 Yeah, I have two full time jobs too. 1452 01:26:42,640 --> 01:26:46,760 It's, you know, building TV labs and being a dad of a three and a 1453 01:26:46,760 --> 01:26:49,000 five year old, it's, it's challenging. 1454 01:26:49,560 --> 01:26:52,160 Yeah, I bet. Do you? 1455 01:26:52,160 --> 01:26:55,200 Do you get to work from home most of the time or do you have 1456 01:26:55,200 --> 01:26:59,040 to go into the office? I split my time between here and 1457 01:26:59,280 --> 01:27:02,040 in Brooklyn. Unfortunately I'm like an hour 1458 01:27:02,040 --> 01:27:06,440 and a half three trains to get there, so I do a lot of coding 1459 01:27:06,440 --> 01:27:09,720 on the train. That's nice, actually. 1460 01:27:09,720 --> 01:27:12,080 Like that, there's a benefit of living in a city. 1461 01:27:12,080 --> 01:27:14,840 You don't have to drive, you can get stuff done on your commute. 1462 01:27:15,800 --> 01:27:19,480 I one of the things I, I just did is I got myself a Mac mini 1463 01:27:19,480 --> 01:27:22,280 and now I'm orchestrating LLMS on the Mac mini. 1464 01:27:22,280 --> 01:27:25,880 So I, I have it on tailscale. I SSH into the box over 1465 01:27:25,880 --> 01:27:28,840 tailscale, I orchestrate the LLMS, and then even as I'm 1466 01:27:28,840 --> 01:27:30,360 crossing trains, they're still running. 1467 01:27:30,880 --> 01:27:35,680 That I should get a Mac mini? Yeah, you should or you should 1468 01:27:35,680 --> 01:27:37,560 just do. The smarter thing is use the the 1469 01:27:37,560 --> 01:27:39,320 cloud agents and just pay for that. 1470 01:27:39,320 --> 01:27:42,320 But I wanted to have it on my own compute. 1471 01:27:44,160 --> 01:27:47,000 Taking notes, say I learned this is my learning time. 1472 01:27:47,000 --> 01:27:49,680 I have cool people on and they teach me things. 1473 01:27:51,680 --> 01:27:53,360 I'm like the same as my listeners. 1474 01:27:53,360 --> 01:27:55,200 I'm like, oh I got a note that for later. 1475 01:27:57,440 --> 01:28:00,120 Love it. Yeah, well, what's like, what's 1476 01:28:00,120 --> 01:28:04,400 funny too is I, I'll, I let YouTube create the transcripts 1477 01:28:04,400 --> 01:28:06,720 for this. It takes like 24 hours about 1478 01:28:06,720 --> 01:28:10,560 after we're done live streaming and then it produces, you know, 1479 01:28:10,560 --> 01:28:13,040 all my time stamps and descriptions for the actual 1480 01:28:13,040 --> 01:28:14,760 podcast. Because then that actually 1481 01:28:14,760 --> 01:28:19,080 covers what we talked about instead of like my 2 sentences 1482 01:28:19,080 --> 01:28:23,720 of what I think we're going to talk about because it it always 1483 01:28:23,720 --> 01:28:27,200 goes down the rabbit holes of. Course, that's the fun part. 1484 01:28:28,440 --> 01:28:31,080 Well, Dave, I think we can wrap this up. 1485 01:28:31,080 --> 01:28:33,480 I really appreciated this conversation. 1486 01:28:33,480 --> 01:28:37,360 It was, it was cool. I you're always welcome on again 1487 01:28:37,760 --> 01:28:39,360 and. Thank you for having me, this is 1488 01:28:39,360 --> 01:28:41,720 really fun. Yeah, and thanks for everyone 1489 01:28:41,720 --> 01:28:44,240 that tuned in. We had a couple comments. 1490 01:28:44,480 --> 01:28:47,760 Sometimes it shoots off a little more and sometimes people I 1491 01:28:47,760 --> 01:28:50,360 think are just so engaged. They're learning so much. 1492 01:28:50,360 --> 01:28:52,320 They. That must be a chat. 1493 01:28:52,840 --> 01:28:57,640 It's definitely a. But awesome, thanks so much for 1494 01:28:57,640 --> 01:29:01,800 joining me and I'm sure I'll have you on again at some point. 1495 01:29:01,800 --> 01:29:08,600 So, yeah, all right, guys, have a good weekend testing.