1 00:00:03,640 --> 00:00:06,680 I'm not going to ask how your windows got fragmented anyway. 2 00:00:10,880 --> 00:00:15,400 Welcome, welcome, Welcome to the Embedded AI Podcast where we 3 00:00:15,400 --> 00:00:20,240 talk about embedded development, embedded systems, and we talk 4 00:00:20,240 --> 00:00:23,640 about AI is artificial intelligence. 5 00:00:23,640 --> 00:00:25,520 For those that you didn't know, it's called artificial 6 00:00:25,520 --> 00:00:27,880 intelligence if you're new to the game. 7 00:00:28,240 --> 00:00:31,800 People have been going all sorts of crazy about it over the past 8 00:00:32,040 --> 00:00:34,560 couple years, even more so in the past couple weeks. 9 00:00:34,560 --> 00:00:38,040 We are not talking about Moat bot and open claw and clawed bot 10 00:00:38,040 --> 00:00:39,520 today. Nope, not on the table. 11 00:00:39,920 --> 00:00:41,720 Maybe in the future, but not today. 12 00:00:42,840 --> 00:00:46,840 I am your host in real life, joining you from the real world. 13 00:00:47,040 --> 00:00:50,520 I'm Ryan Torbic, joined by. Lukaijani. 14 00:00:50,960 --> 00:00:52,480 Yes. And we're here today. 15 00:00:52,480 --> 00:00:55,440 We're going to talk we're while we're the entire podcast we're 16 00:00:55,440 --> 00:00:58,400 going to cover is we're covering all that overlap between 17 00:00:58,400 --> 00:01:01,360 embedded systems and AI development and AI, how you're 18 00:01:01,360 --> 00:01:04,959 using AI and embedded systems and large language models and 19 00:01:04,959 --> 00:01:07,000 not large language models and all that kind of stuff. 20 00:01:07,000 --> 00:01:09,840 Today we're going to be talking about, let's see, what are we 21 00:01:09,840 --> 00:01:12,760 going to talk about? Oh yes, we're going to talk 22 00:01:12,760 --> 00:01:15,960 about developing embedded systems and having the LLM 23 00:01:15,960 --> 00:01:19,000 actually debug the embedded system with us. 24 00:01:19,120 --> 00:01:21,040 Not for us. I'm going to clear it not for 25 00:01:21,040 --> 00:01:25,720 us, but with us. So Oka, I think you the most 26 00:01:25,720 --> 00:01:28,200 recent experience with this. I can I can go into, but I'm 27 00:01:28,200 --> 00:01:30,280 going to be a lot of yelling, so I'm going to let you started 28 00:01:30,280 --> 00:01:36,600 like. I've I've everybody who's 29 00:01:36,600 --> 00:01:39,040 programming has the most recent experience with debugging, I 30 00:01:39,040 --> 00:01:42,400 guess, but but me maybe more than others. 31 00:01:42,400 --> 00:01:44,160 OK, maybe on to something. There you go. 32 00:01:45,520 --> 00:01:50,240 See. No, but it is interesting to, to 33 00:01:50,240 --> 00:01:55,360 think about how you can make best use of LLMS and of modern 34 00:01:55,760 --> 00:02:00,640 AI powered coding tools, not just as you are, you know, 35 00:02:00,800 --> 00:02:04,880 writing features and, and all of that, but also while you are 36 00:02:05,160 --> 00:02:08,919 trying to get them to actually work the way you want them to. 37 00:02:11,080 --> 00:02:14,760 And you know, it, it makes sense to, to have the LLM help you 38 00:02:14,760 --> 00:02:17,520 there. And I think I, I think, but it's 39 00:02:17,520 --> 00:02:20,360 systems are a bit of a special case because you know, they, 40 00:02:20,520 --> 00:02:22,920 they've got this pesky thing called hardware that's, that 41 00:02:22,920 --> 00:02:26,920 tends to be involved. And they are just, they tend to 42 00:02:26,920 --> 00:02:29,960 be higher complexity, even simple ones because of that. 43 00:02:30,840 --> 00:02:34,640 And so maybe it makes sense to talk about some tricks that 44 00:02:34,640 --> 00:02:37,960 we've learned, some observations that we've made, some mistakes 45 00:02:37,960 --> 00:02:41,840 that we've made. I'm telling you, the board was 46 00:02:41,840 --> 00:02:44,240 not smoking. It just had an aroma that I was 47 00:02:44,240 --> 00:02:52,640 like, I think it's smoking. You let this make it magic smoke 48 00:02:52,640 --> 00:02:55,480 out, did you? But I was like, I don't see any 49 00:02:55,480 --> 00:02:57,200 smoke. All the cables look right. 50 00:02:58,240 --> 00:03:00,640 Oh, it's kind of snow, actually. Let's let's stop that. 51 00:03:00,760 --> 00:03:02,400 From the oven like it's the oven on. 52 00:03:03,560 --> 00:03:06,040 No, but let's start there because this is an interesting 53 00:03:06,040 --> 00:03:09,280 one. Like obviously in a better 54 00:03:09,280 --> 00:03:13,040 systems you have like you have electric circuits and you have 55 00:03:13,480 --> 00:03:16,480 wirings that you that you need to set up. 56 00:03:16,520 --> 00:03:20,080 And of course you can be a smart person or you can be me and sort 57 00:03:20,080 --> 00:03:25,040 of mix up ground and VCC. You can do that. 58 00:03:25,040 --> 00:03:26,680 It's just going different directions, right? 59 00:03:26,680 --> 00:03:30,640 It's fine like you. Know, but the point is. 60 00:03:30,800 --> 00:03:35,840 The point is, a while ago I was, I was trying to get a, a simple 61 00:03:36,280 --> 00:03:39,560 circuit to work on, on a breadboard and it just, it just 62 00:03:39,560 --> 00:03:41,920 wouldn't work. It, it, it was just not, not 63 00:03:41,920 --> 00:03:47,040 going very well. And I actually took a picture 64 00:03:47,280 --> 00:03:51,120 off the breadboard and, and showed it to Claude Codd. 65 00:03:51,120 --> 00:03:54,360 I said, well, can you spot anything like is there, is there 66 00:03:54,360 --> 00:03:56,000 something that's wrong with this circuit? 67 00:03:56,760 --> 00:04:01,880 And it found it and it was exactly what I, what I, what I 68 00:04:01,880 --> 00:04:06,280 said, I, I, I should have, I should have pulled, I should 69 00:04:06,280 --> 00:04:08,560 have pulled the signal line up, but I was pulling it down. 70 00:04:09,240 --> 00:04:11,160 Oh, I see, I see. Gotcha. 71 00:04:11,160 --> 00:04:13,920 OK, oops. And it figured that out for you. 72 00:04:15,120 --> 00:04:19,200 Yes, it did. Well, I, I had kind of realized 73 00:04:19,200 --> 00:04:21,760 that by myself, but, but I wanted to see whether it would 74 00:04:21,800 --> 00:04:23,760 also find it. And yes, it did find it. 75 00:04:24,280 --> 00:04:29,040 And, and it was also useful because we were debugging the 76 00:04:29,040 --> 00:04:31,160 software because first we thought maybe it's a timing 77 00:04:31,160 --> 00:04:33,640 issue, you know, maybe it's, maybe it's a software problem. 78 00:04:34,120 --> 00:04:40,640 And what, and, and the, the, the circuit error was consistent 79 00:04:40,640 --> 00:04:44,040 with the kind of behaviour we saw in the software. 80 00:04:44,680 --> 00:04:48,000 That is, you know, that, that the sensor appeared to be alive, 81 00:04:48,000 --> 00:04:51,280 but it claimed to be, I think uninitialized or something. 82 00:04:51,560 --> 00:04:54,320 It turns out that's just a quirk of the, of the actual protocol. 83 00:04:54,320 --> 00:04:58,800 And if it's just pulled down, then then if the protocol 84 00:04:58,800 --> 00:05:02,520 assumes that the that the sensor is actually there, which in a 85 00:05:02,520 --> 00:05:04,800 sense it isn't, but it's not not at all working. 86 00:05:05,120 --> 00:05:07,120 Yeah, yeah. Yeah. 87 00:05:07,120 --> 00:05:08,720 But this, this was really interesting. 88 00:05:08,720 --> 00:05:14,200 Like this is such a fascinating trick to take a picture and show 89 00:05:14,200 --> 00:05:16,880 it to the LLM and and at least that way you can sort of reason 90 00:05:16,880 --> 00:05:21,600 through with the LLM because even if it hadn't spotted it 91 00:05:21,600 --> 00:05:24,560 right away, we could have said, OK, fine, let's let's go over 92 00:05:24,560 --> 00:05:27,160 the data sheet. What you know didn't connect the 93 00:05:27,160 --> 00:05:30,440 pins correctly And yes, I can do all of that by hand obviously, 94 00:05:30,440 --> 00:05:33,800 but the LLM is just much quicker than I am. 95 00:05:33,800 --> 00:05:37,120 It's like, OK, fine, let me see what are the correct voltages? 96 00:05:37,360 --> 00:05:39,080 How should all of this be connected? 97 00:05:39,640 --> 00:05:43,920 Or you know it, it can decode the the resistor color codes 98 00:05:43,920 --> 00:05:47,720 much quicker than I can. Oh wait, you got to do that. 99 00:05:47,920 --> 00:05:51,360 Oh man, I don't know. Resistors were not made for 100 00:05:51,360 --> 00:05:53,280 color blind people. I just want to say right. 101 00:05:56,000 --> 00:06:00,400 You're color blind. I I'm not much of A hardware 102 00:06:00,400 --> 00:06:02,240 person. My digital audit design course, 103 00:06:02,240 --> 00:06:04,440 they're like, oh, and then make sure it's got three Reds and a 104 00:06:04,440 --> 00:06:06,240 green. I'm like, I hate you guys. 105 00:06:09,040 --> 00:06:12,240 Can you just write, write it on there like really small, like 106 00:06:12,320 --> 00:06:13,640 why? It was all the label. 107 00:06:13,640 --> 00:06:15,760 OK, that's fine. That's great. 108 00:06:16,560 --> 00:06:19,880 Yeah. No so, but maybe that's a first 109 00:06:19,880 --> 00:06:23,680 interesting trick to to really. I literally, I literally was, 110 00:06:23,680 --> 00:06:26,040 I'd taken a resistor and I was like, I don't know which one 111 00:06:26,040 --> 00:06:28,160 this is. So I had my like little sheets 112 00:06:28,160 --> 00:06:31,640 of resistors and they all had like the 10K and the 220 and I 113 00:06:31,640 --> 00:06:34,280 was, and so I was literally taking them and comparing them 114 00:06:34,280 --> 00:06:37,400 next to each of the pile to be like, which one is this? 115 00:06:39,400 --> 00:06:41,400 And to be able to hand that off to an LM would be great. 116 00:06:41,440 --> 00:06:44,560 Because my other solution is to find a normal person who's not 117 00:06:44,560 --> 00:06:47,160 color blind and have them help me, but they're not in my house. 118 00:06:47,160 --> 00:06:50,160 And so then I'm taking pictures or going on a a video call and 119 00:06:50,160 --> 00:06:52,120 then making sure the light is right and like. 120 00:06:57,840 --> 00:07:01,360 Yeah, I, I feel like in in many cases, those kinds of tricks 121 00:07:01,360 --> 00:07:04,560 aren't really about like how how good is the LLM or something, 122 00:07:04,560 --> 00:07:06,320 but rather it's a matter of your fantasy. 123 00:07:06,320 --> 00:07:10,160 Like, do you even come up with the idea of, well, I could show 124 00:07:10,160 --> 00:07:12,040 this to the LLM, but it would be so much easier? 125 00:07:12,400 --> 00:07:16,480 Yes, yeah, the well, in Cloud Code, actually, after you told 126 00:07:16,480 --> 00:07:18,480 me that cloud, the next time I was using Cloud Code, it 127 00:07:18,480 --> 00:07:21,040 literally printed out like a little, it's giving you these 128 00:07:21,040 --> 00:07:23,680 tips now, these little hints while it's processed. 129 00:07:23,680 --> 00:07:28,080 And it's like, did you know you can control V an image into the 130 00:07:28,080 --> 00:07:29,440 buffer? And then I'll read it for you 131 00:07:29,440 --> 00:07:32,840 and I'm like, what? All right, look, just did this. 132 00:07:34,440 --> 00:07:37,160 Yeah, it it. It still feels strange to copy 133 00:07:37,160 --> 00:07:41,640 an image into a a console window, but anyway, especially 134 00:07:41,640 --> 00:07:44,680 fine. You can even drag and drop an 135 00:07:44,680 --> 00:07:46,480 image in there. And. 136 00:07:47,480 --> 00:07:50,720 It works and I I use this even outside the but it's systems 137 00:07:50,720 --> 00:07:53,240 context. For example when I work on the 138 00:07:53,240 --> 00:07:57,320 website for the Embedded Eye podcast because I am utterly 139 00:07:57,320 --> 00:08:00,080 clueless about web development. Yeah. 140 00:08:00,520 --> 00:08:04,800 So if the site doesn't look the way I want it to, I very often 141 00:08:04,800 --> 00:08:08,720 just take a screenshot, show it to Cloud Code and say, you know, 142 00:08:08,880 --> 00:08:12,000 center this thing for me or, you know, make it pretty or 143 00:08:12,200 --> 00:08:14,440 something. Something I think there's 144 00:08:14,440 --> 00:08:18,000 there's well, and this kind of like graphics is is is 145 00:08:18,000 --> 00:08:21,040 interesting because I think a lot of the the examples that 146 00:08:21,040 --> 00:08:25,280 I've seen are all about web development where nothing ever 147 00:08:25,280 --> 00:08:28,440 leaves the the desktop. Everything is going to be on the 148 00:08:28,440 --> 00:08:30,760 desktop or on the screen of the, of the computer. 149 00:08:30,760 --> 00:08:34,360 And so, you know, when you think about interacting with a piece 150 00:08:34,360 --> 00:08:39,159 of hardware, like it's the data's left the system, like 151 00:08:39,360 --> 00:08:42,120 the, the LOM doesn't have access to it anymore because it's gone 152 00:08:42,120 --> 00:08:45,480 over a USB cable and it's now Ors and flashed to a, a chip, a 153 00:08:45,480 --> 00:08:47,680 card and they've plugged in. Like there's just, it's not 154 00:08:47,680 --> 00:08:49,040 there anymore what's running on there. 155 00:08:49,040 --> 00:08:53,280 So like trying to get that feedback into the LOM is really 156 00:08:53,280 --> 00:08:55,000 difficult, especially when you're dealing with physical 157 00:08:55,000 --> 00:08:57,080 systems. And so being able to take a 158 00:08:57,080 --> 00:08:59,680 photo, I had this, I, I still have this dream and I'm like, 159 00:08:59,840 --> 00:09:01,840 oh, I don't know if I should like pursue this or not. 160 00:09:02,120 --> 00:09:05,120 But like putting a webcam on a piece of hardware and then 161 00:09:05,120 --> 00:09:07,800 having it flash the hardware and then like feeding the webcam 162 00:09:07,800 --> 00:09:11,600 data into the LOLM so it can watch it can watch with my with 163 00:09:11,840 --> 00:09:15,480 with eyes. It's a weird thing to say, but 164 00:09:15,480 --> 00:09:18,040 like to have a webcam on the hardware and then be able to 165 00:09:18,040 --> 00:09:20,560 pull that data and actually be able to watch it blink or watch 166 00:09:20,560 --> 00:09:22,440 it turn or watch it rotate and be able to like. 167 00:09:23,080 --> 00:09:25,080 So I don't know if how I feel about that. 168 00:09:25,080 --> 00:09:26,400 I don't know if I want to go there yet. 169 00:09:26,440 --> 00:09:29,720 That's we're definitely not there yet, but that might be 170 00:09:29,720 --> 00:09:32,680 somewhere we go in the future because this this interacting 171 00:09:32,680 --> 00:09:35,440 with the physical world is, is outside the realm. 172 00:09:35,440 --> 00:09:37,600 And so like being able to pull a picture off and then give it to 173 00:09:37,600 --> 00:09:41,040 it's like, whoa, OK, it does get thing wrong. 174 00:09:41,400 --> 00:09:45,680 I'll say that. Yeah, but I mean, if, if, if you 175 00:09:45,880 --> 00:09:49,400 are showing still pictures, going to moving pictures is not, 176 00:09:49,520 --> 00:09:52,000 not that much of A stretch. Like it could be a sequence of 177 00:09:52,000 --> 00:09:54,640 still pictures. And in fact, like for, for my 178 00:09:54,640 --> 00:09:59,400 embedded systems trainings, I actually, I actually have a 179 00:09:59,400 --> 00:10:02,440 camera set up because I, I don't want to use simulator. 180 00:10:02,440 --> 00:10:06,040 So I, I'll have real breadboards with real microcontrollers on 181 00:10:06,040 --> 00:10:10,560 them where my students, you know, work on whatever they're 182 00:10:10,560 --> 00:10:12,680 working on. And so that they can see what's 183 00:10:12,680 --> 00:10:17,320 happening, I have a camera trained on on the breadboards so 184 00:10:17,320 --> 00:10:21,320 that they can observe whether, you know, whether the LEDs are 185 00:10:21,320 --> 00:10:23,680 in fact blinking or not. And that's something, yeah. 186 00:10:24,240 --> 00:10:27,280 And it would be trivial to get the LLM to washed out as well. 187 00:10:27,760 --> 00:10:29,720 Right. But then what do I do with it? 188 00:10:29,760 --> 00:10:33,600 Yeah, Anyways, so I will say so I was, I was wiring up. 189 00:10:33,640 --> 00:10:36,440 That was my my the board smells funny. 190 00:10:36,440 --> 00:10:41,120 The other day I was wiring it up and the sensor I was using was a 191 00:10:41,120 --> 00:10:43,920 3.3 Volt sensor. And so I was like, I plugged it 192 00:10:43,920 --> 00:10:46,240 into the I, I don't have an Arduino. 193 00:10:46,240 --> 00:10:49,800 I have an elegue, but it's, you know, it's the same thing. 194 00:10:51,280 --> 00:10:57,080 So I, so I hooked it up. I had this RFID sensor hooked 195 00:10:57,080 --> 00:10:59,040 up. I had all the, the cables coming 196 00:10:59,040 --> 00:11:01,640 out of it. And then they have all these 197 00:11:01,640 --> 00:11:06,480 different the MISO, the MOSI and the, you know, the reset pin all 198 00:11:06,480 --> 00:11:09,640 all the pins set up correctly 3.3 Volt yes. 199 00:11:09,640 --> 00:11:12,520 I checked it 100 times. The 3.3 is 3.3. 200 00:11:12,600 --> 00:11:14,840 Got up my glasses to look at it really small. 201 00:11:15,000 --> 00:11:19,160 The grounds over here like yes, yes, yes, I had the LM actually 202 00:11:19,160 --> 00:11:22,640 printing me circuit diagrams. So give me an actual circuit 203 00:11:22,640 --> 00:11:25,960 diagram of what it's supposed to look like so I can I can build 204 00:11:25,960 --> 00:11:28,800 the the circuit that that you said to do it and it was printed 205 00:11:28,800 --> 00:11:30,680 out. I had it do mermaid ones in 206 00:11:30,680 --> 00:11:34,280 markdown and actually put an SVG together that it dumped out and 207 00:11:34,280 --> 00:11:36,360 and it it had the circuit diagram on it like this is 208 00:11:36,360 --> 00:11:38,440 connected this these are the cables, this is the pins, all 209 00:11:38,440 --> 00:11:41,240 that stuff. And so like, wow, that's really 210 00:11:41,240 --> 00:11:42,280 cool. So I was following the 211 00:11:42,280 --> 00:11:45,000 instructions that it gave me did this whole thing started to 212 00:11:45,000 --> 00:11:47,000 smell a little bit, but I was like, but it's said to do this. 213 00:11:47,320 --> 00:11:48,600 Nothing was working. Nothing was working. 214 00:11:48,600 --> 00:11:50,440 It continued to be uninitialized the whole time. 215 00:11:50,640 --> 00:11:53,960 I finally decided like, OK, I'll take a photos, do the photos. 216 00:11:54,280 --> 00:11:59,440 Took a photo of the RFID sensor with all the the cables coming 217 00:11:59,440 --> 00:12:02,400 out of it. And well, so the first I tried 218 00:12:02,400 --> 00:12:04,520 it on a breadboard and then I read somewhere because I started 219 00:12:04,520 --> 00:12:07,000 cheating by not, I started cheating on the LLM by actually 220 00:12:07,000 --> 00:12:14,840 using the Internet. So, so I, I looked at, I, I 221 00:12:14,880 --> 00:12:16,160 looked, was starting to look at forms. 222 00:12:16,160 --> 00:12:18,200 So I had it into the breadboard and I took a picture of the 223 00:12:18,200 --> 00:12:19,760 breadboard and it was like, yeah, it looks fine. 224 00:12:19,760 --> 00:12:24,520 I'm like, OK, sure, let's let's try again. 225 00:12:24,520 --> 00:12:26,240 So then I was started reading more stuff on the Internet and 226 00:12:26,240 --> 00:12:27,840 they said, oh, well, I couldn't get it to work with the 227 00:12:27,840 --> 00:12:28,960 breadboard. So I was like, OK, cool. 228 00:12:28,960 --> 00:12:30,840 So I ripped it all apart, put it back together. 229 00:12:31,000 --> 00:12:33,600 And then I was taking photos and I, it had different colored 230 00:12:33,600 --> 00:12:36,600 wires coming out of it. And I said, can you tell me if 231 00:12:36,600 --> 00:12:39,080 these are hooked up correctly? So I took a picture of the wires 232 00:12:39,080 --> 00:12:41,320 coming out of it. And then I took a picture of 233 00:12:41,320 --> 00:12:44,480 the, the header on the Arduino and said, now look at this 234 00:12:44,480 --> 00:12:46,120 picture. Are these colors the same? 235 00:12:46,400 --> 00:12:49,360 And it really could not, it could not look at one picture 236 00:12:49,360 --> 00:12:52,160 and then it couldn't look at and then take that information and 237 00:12:52,160 --> 00:12:54,920 look at the other picture and say, yes, these are equivalent. 238 00:12:55,040 --> 00:12:57,200 It kept telling me you can't plug this straight in. 239 00:12:57,200 --> 00:12:58,320 They have to be in a different order. 240 00:12:58,320 --> 00:13:01,080 I'm like, I know are they? I did. 241 00:13:01,080 --> 00:13:04,560 I did that. So it was really interesting to 242 00:13:04,560 --> 00:13:06,960 see, like the limitation. And it never actually gave. 243 00:13:06,960 --> 00:13:08,200 It just kept telling me the same thing. 244 00:13:08,200 --> 00:13:10,400 It's like, yeah, you need to move them in these order. 245 00:13:10,400 --> 00:13:12,880 I'm like, yeah, I'm color blind. Help me out here man. 246 00:13:15,840 --> 00:13:19,920 Yeah, yeah. That's, that's sort of a risk 247 00:13:20,000 --> 00:13:22,240 in, in debugging together with an LLM. 248 00:13:22,240 --> 00:13:25,920 Because if it's not, if something is not working, then 249 00:13:25,920 --> 00:13:31,400 the LLM will very easily go down, you know, a strange path 250 00:13:31,440 --> 00:13:34,520 and then just sort of be stuck there and sort of oscillate 251 00:13:34,520 --> 00:13:38,160 between two faulty states or, or sort of dig itself ever deeper 252 00:13:38,160 --> 00:13:40,000 into a hole. Which is, which is one of the 253 00:13:40,000 --> 00:13:42,880 reasons why I'm still not convinced that fully autonomous 254 00:13:42,880 --> 00:13:47,760 coding is actually works. Despite, you know, what all the 255 00:13:47,760 --> 00:13:50,520 cool people on YouTube claim. I, I, I simply don't believe it. 256 00:13:53,040 --> 00:13:57,480 Also because they tend to make the same kinds of mistakes and 257 00:13:57,480 --> 00:13:59,240 the same kinds of simplifications. 258 00:14:00,680 --> 00:14:05,320 I showed Ryan just today how I pretty much yelled at an LLM 259 00:14:05,320 --> 00:14:08,840 because it, it keeps, you know, if a test is failing, it keeps 260 00:14:08,840 --> 00:14:13,120 softening the test to make it pass instead of realizing, oh, 261 00:14:13,640 --> 00:14:16,040 it just showed me a bug, let me go fix the bug. 262 00:14:16,040 --> 00:14:19,240 And it's like, Oh no, no, let me, let me make the test green. 263 00:14:20,000 --> 00:14:23,880 And it's so aggravating. Basically I I cannot get it to 264 00:14:24,760 --> 00:14:28,560 be sensitive about this. It's terrible. 265 00:14:30,120 --> 00:14:34,680 Anyway, I made the test pass apparently. 266 00:14:34,880 --> 00:14:36,640 Fantastic. The LLM is not tested here. 267 00:14:36,680 --> 00:14:40,040 Yeah. So anyway, so I, I would be very 268 00:14:40,040 --> 00:14:46,120 careful to give an LLM too much space to roam, but it can be 269 00:14:46,120 --> 00:14:50,560 extremely helpful both in terms of, you know, just simplifying, 270 00:14:50,560 --> 00:14:54,040 you know, reading colour codes of resistors for you or, or 271 00:14:54,040 --> 00:14:58,160 summarizing a data sheet or, or, you know, matching what's in a 272 00:14:58,160 --> 00:15:01,280 data sheet to what's in your init function, let's say. 273 00:15:02,680 --> 00:15:05,200 But also it is, it is a fantastic rubber duck. 274 00:15:06,800 --> 00:15:09,160 You know about rubber duck debugging, right where you, you 275 00:15:09,160 --> 00:15:13,760 tell traditionally an inanimate object like a rubber duck about 276 00:15:13,760 --> 00:15:16,400 your problem. And through the process of 277 00:15:17,040 --> 00:15:21,440 expressing the situation, very often you realise, you know, oh 278 00:15:21,840 --> 00:15:25,560 you know, the problem is actually problem exists between 279 00:15:25,560 --> 00:15:31,400 keyboard and chair. Please replace the entity 280 00:15:31,400 --> 00:15:32,680 between the keyboard and the chair. 281 00:15:33,600 --> 00:15:38,120 Yes, indeed. And and that's just really 282 00:15:38,120 --> 00:15:42,000 practical because quite often it will, especially in trickier 283 00:15:42,000 --> 00:15:44,680 cases, you know, it can't actually figure out the problem 284 00:15:44,680 --> 00:15:50,160 on its own, but it will give you enough feedback and enough sort 285 00:15:50,160 --> 00:15:55,000 of scaffolding to hang your thoughts on that you can spot 286 00:15:55,000 --> 00:15:57,640 the problem on your own much easier because we'll say, oh, 287 00:15:57,640 --> 00:15:59,520 yeah, the typical approach is this, blah, blah, blah, blah, 288 00:15:59,520 --> 00:16:01,640 blah, blah, blah. And then you realize, OK, half 289 00:16:01,640 --> 00:16:04,160 of the approach is actually, you know, pointless because it got 290 00:16:04,160 --> 00:16:06,160 wrong. But, oh, here's a thing that I 291 00:16:06,160 --> 00:16:07,880 haven't checked yet. Yeah. 292 00:16:08,280 --> 00:16:10,600 Then you go and check it. Then it turns out maybe it was 293 00:16:10,600 --> 00:16:11,960 that. So. 294 00:16:12,080 --> 00:16:17,080 So. So along those same lines, I 295 00:16:17,080 --> 00:16:20,000 have noticed like printf debugging, which is which is 296 00:16:20,000 --> 00:16:22,800 still very standard and especially in embedded systems 297 00:16:22,800 --> 00:16:26,640 development, printf debugging over you are is you know, it, 298 00:16:26,680 --> 00:16:31,080 it, it handles it very well. And I have watched the LL and 299 00:16:31,080 --> 00:16:34,080 this is something that you know, me as a person who is, you know, 300 00:16:34,080 --> 00:16:38,920 physically typed the code to go through and then delete all of 301 00:16:38,920 --> 00:16:43,560 the code and replace it with a much simpler, you know, like, 302 00:16:43,720 --> 00:16:46,920 OK, you know what this is this complex system we've built is 303 00:16:46,920 --> 00:16:49,600 not working. Let me just create something 304 00:16:49,600 --> 00:16:54,160 really simple to do that and like the, the mental effort and 305 00:16:54,160 --> 00:16:57,600 the like, when I've done this, I, I mean, I'll, I'll go there, 306 00:16:57,600 --> 00:17:00,400 but I'll definitely like, OK, let me, let me see how I can 307 00:17:00,400 --> 00:17:03,680 short circuit this so that I, I, it goes to this simple case as 308 00:17:03,680 --> 00:17:06,280 opposed to I've watched the LLM completely delete everything 309 00:17:06,280 --> 00:17:09,200 that is done, replace it with the base case. 310 00:17:09,200 --> 00:17:11,720 Like, OK, I, I think there's something wrong with all of our 311 00:17:11,720 --> 00:17:14,160 assumptions. Replace it with the base case 312 00:17:14,839 --> 00:17:16,520 and there's not. And it was like that it was, it 313 00:17:16,520 --> 00:17:19,319 was not, it was not like, OK, well, it's going to take me, you 314 00:17:19,319 --> 00:17:21,960 know, 1/2 hour to go and do this or 10 minutes or whatever. 315 00:17:22,119 --> 00:17:25,839 No, it just deleted everything, put put the base case in there 316 00:17:25,920 --> 00:17:28,840 and then then it tested and then it ran, it tested against the 317 00:17:28,840 --> 00:17:31,800 base case. Like, OK, I see, I see this. 318 00:17:32,080 --> 00:17:34,720 And then build it back up and then actually build up the whole 319 00:17:34,720 --> 00:17:38,120 solution without me having to go through and do all the typing. 320 00:17:38,480 --> 00:17:41,240 That was fascinating to me to see it actually go through that 321 00:17:41,240 --> 00:17:44,560 process and so quickly because the number of times I've gone 322 00:17:44,560 --> 00:17:47,280 through, I'm like, OK, OK, I'm at C++. 323 00:17:47,280 --> 00:17:50,800 I got to include IO stream in this file and then I got STDC 324 00:17:50,800 --> 00:17:52,680 out. Oh, do I use namespace STD? 325 00:17:52,680 --> 00:17:54,680 And then I'm having to go and copy paste and I have to put a 326 00:17:54,680 --> 00:17:56,800 meaningful message in each one. It's got to be different. 327 00:17:56,800 --> 00:17:59,240 So I know which stage it's going to, Oh, now it's going to this 328 00:17:59,240 --> 00:18:00,520 function. OK, now go to that file. 329 00:18:00,720 --> 00:18:03,720 Like the amount of time that it's taken me to do that and to 330 00:18:03,720 --> 00:18:06,040 watch the lol, just go through that in seconds. 331 00:18:06,160 --> 00:18:08,200 It was just like, and it was the same. 332 00:18:08,200 --> 00:18:10,200 It would be the same thing. I was like wow. 333 00:18:10,840 --> 00:18:11,800 That's cool. Yeah. 334 00:18:12,000 --> 00:18:13,720 And it's, and it's also the same thing. 335 00:18:13,800 --> 00:18:17,480 So there's actually two things at play here. 1 is of course 336 00:18:17,480 --> 00:18:20,600 it's much quicker just adding a couple of print of statements 337 00:18:20,800 --> 00:18:22,880 and also it's much quicker at analysing the output. 338 00:18:22,880 --> 00:18:25,160 Then I say ah, OK, there we go there we go we see what we 339 00:18:25,160 --> 00:18:27,040 wanted to see or Oh no, there's a gap in here. 340 00:18:27,560 --> 00:18:30,080 So that is very convenient because it, it really, really 341 00:18:30,080 --> 00:18:33,680 tightens this feedback loop. Yeah, which is always very 342 00:18:33,680 --> 00:18:35,840 valuable. But also it has another effect, 343 00:18:35,840 --> 00:18:37,480 which is that is that a bug you? Is that? 344 00:18:37,480 --> 00:18:42,120 A plug for the Agile Embedded Podcast Tight Feedback Loops. 345 00:18:42,120 --> 00:18:42,800 Boo. Yeah. 346 00:18:42,800 --> 00:18:45,960 Oh yeah. I mean, it is the truth though, 347 00:18:45,960 --> 00:18:48,760 isn't it? It is so. 348 00:18:49,320 --> 00:18:52,880 But the other great thing about this is that you, the human, can 349 00:18:52,880 --> 00:18:56,000 sort of take a step back and consider the bigger picture. 350 00:18:56,000 --> 00:19:01,040 You know, while, while the LLM is busy putting actual print of 351 00:19:01,040 --> 00:19:04,400 statements into like actual files, you can sort of step back 352 00:19:04,400 --> 00:19:08,760 and say, well, which module should actually be the part of 353 00:19:08,760 --> 00:19:13,320 whatever the problem is? And like, do I see the flow of 354 00:19:13,320 --> 00:19:16,440 printfs that I was expecting and those kinds of things. 355 00:19:16,440 --> 00:19:20,440 So sort of on a much higher level, you sort of think about 356 00:19:20,760 --> 00:19:24,440 what is it that you're observing versus what you were expecting 357 00:19:24,440 --> 00:19:27,720 to observe? And that tends to be extremely 358 00:19:27,720 --> 00:19:29,480 powerful. And I mean, this is one of those 359 00:19:29,480 --> 00:19:34,320 things about AI in general, it, it tends to be that humans are 360 00:19:34,320 --> 00:19:36,560 much better at the, at the larger scale, under the broad 361 00:19:36,560 --> 00:19:39,000 context. And LLMS are much better at the, 362 00:19:39,200 --> 00:19:42,040 you know, tiny sort of code line level stuff. 363 00:19:43,280 --> 00:19:45,880 And if you combine those strengths, then then you just 364 00:19:45,880 --> 00:19:49,520 make such big strides because the, the LLM will be, will be 365 00:19:49,520 --> 00:19:50,920 lost in a big context without you. 366 00:19:51,320 --> 00:19:54,560 But like you say, you know, changing all of those printers 367 00:19:54,560 --> 00:19:57,560 and is going to take forever, whereas the LLM doesn't care. 368 00:19:57,560 --> 00:20:02,360 Like it it, it doesn't, you know, it's, it's as I, as I like 369 00:20:02,360 --> 00:20:06,960 to say, the overeager intern. It's, it's a little naive and 370 00:20:06,960 --> 00:20:12,040 it's a little, you know, but, but you know, it can, it can 371 00:20:12,040 --> 00:20:14,840 type like the devil. And and it has no emotional 372 00:20:14,840 --> 00:20:17,960 content, no emotional connection to the code that is typed. 373 00:20:17,960 --> 00:20:22,360 Like I don't think people understand how emotional that 374 00:20:22,360 --> 00:20:24,200 connection is. Like, oh, I got to delete all 375 00:20:24,200 --> 00:20:27,280 this code I just wrote. You know, the, the number of 376 00:20:27,280 --> 00:20:29,280 times it's like, you know what, I'm just going to go home 377 00:20:29,280 --> 00:20:31,600 because I know tomorrow I'm going to have to delete all the 378 00:20:31,600 --> 00:20:35,280 stuff I wrote today. Like that, that, that emotional 379 00:20:35,840 --> 00:20:38,920 like poll to be proud of what you've created. 380 00:20:38,920 --> 00:20:42,200 The LLM does not have that. Yeah. 381 00:20:44,240 --> 00:20:47,120 But yeah, but it's interesting that you say that because 382 00:20:47,120 --> 00:20:48,800 apparently different people are different there. 383 00:20:48,800 --> 00:20:52,720 Like I'm, I'm also resistant to or hesitant to delete code that 384 00:20:52,720 --> 00:20:54,880 I wrote, but mostly because I know how much effort was 385 00:20:54,880 --> 00:20:57,960 involved in creating it. It's not that I'm proud of what 386 00:20:57,960 --> 00:20:59,240 I created there because it's buggy. 387 00:20:59,240 --> 00:21:01,040 So like, what is there to be proud of? 388 00:21:01,360 --> 00:21:06,760 But I know that it took me like half a day to painstakingly put 389 00:21:06,760 --> 00:21:09,680 this together and now you want me to just RIP it out now? 390 00:21:10,200 --> 00:21:16,200 No, I, I, you know, it's, it's the same as when I'm riding my 391 00:21:16,200 --> 00:21:19,120 bike. I feel a lot more reluctant to 392 00:21:19,120 --> 00:21:23,720 break because it takes me actual effort to go back to get back up 393 00:21:23,720 --> 00:21:25,000 to speed. Up to speed, yeah. 394 00:21:25,000 --> 00:21:29,080 Whereas in yeah, whereas in the car, like, you know, what does 395 00:21:29,080 --> 00:21:30,480 it matter whether I slow down or not? 396 00:21:30,480 --> 00:21:32,920 Because if I want to go faster, I just press on the loud pedal 397 00:21:32,920 --> 00:21:39,440 and and there we go. Yes, yes, absolutely. 398 00:21:39,600 --> 00:21:43,920 And I don't know this, this. I actually also don't feel the 399 00:21:43,920 --> 00:21:47,640 emotional now I've been working on this is kind of a tangent, 400 00:21:47,640 --> 00:21:51,600 but I've been working on, you know, building up skills in the, 401 00:21:52,400 --> 00:21:57,200 in my coding agents and I will delete all of the code that the 402 00:21:57,200 --> 00:22:00,560 agent has written. I just but like to delete that 403 00:22:00,560 --> 00:22:03,120 skill that I just built. I'm like, I'm so worried that 404 00:22:03,120 --> 00:22:06,040 I'm going to lose this, this very specific set of 405 00:22:06,040 --> 00:22:09,040 instructions that I've made. I better have copies of this 406 00:22:09,040 --> 00:22:11,640 backed up somewhere else. Like I better push this change 407 00:22:11,640 --> 00:22:14,680 to get right now. And so like, I still have that 408 00:22:14,840 --> 00:22:17,120 emotional connection, like I know what it, what the, what it 409 00:22:17,120 --> 00:22:20,880 took to build this up and the steps that I had to take and 410 00:22:20,880 --> 00:22:26,080 the, the, what the, the mistakes that I made in order to make 411 00:22:26,080 --> 00:22:28,320 this work. I still feel that. 412 00:22:28,400 --> 00:22:31,040 And so, but the, the code itself, I don't care anymore. 413 00:22:31,040 --> 00:22:33,440 Like whatever, delete all the code, I can regenerate. 414 00:22:33,440 --> 00:22:35,600 If I already have the prompt that I used, I can regenerate 415 00:22:35,600 --> 00:22:38,240 the whole thing. And so it's like, I don't know, 416 00:22:38,400 --> 00:22:40,280 it's, it's, it's very interesting. 417 00:22:40,680 --> 00:22:44,400 So, but having the the AI go through and do that kind of 418 00:22:44,400 --> 00:22:48,280 manual debugging and have it put in those print statements in and 419 00:22:48,280 --> 00:22:51,640 then reading the output from from the UR, what else we got? 420 00:22:51,680 --> 00:22:57,680 What else we got? Yeah, so I feel like a lot of AI 421 00:22:58,280 --> 00:23:02,280 development centers around around context and complex 422 00:23:02,280 --> 00:23:03,360 management. Yeah. 423 00:23:04,480 --> 00:23:07,160 And interestingly, I think the same is true of debugging. 424 00:23:07,160 --> 00:23:10,200 Like you need to figure out what what actually goes into into the 425 00:23:10,200 --> 00:23:11,960 bug. But the point I was going to 426 00:23:11,960 --> 00:23:17,480 make is it's just really helpful to take data sheets, for 427 00:23:17,480 --> 00:23:20,920 instance, and stick it into the LLM and say, well, here's what 428 00:23:20,920 --> 00:23:23,360 you need to know about this sensor, for instance. 429 00:23:23,480 --> 00:23:27,960 Yeah. And that will help the LLM sort 430 00:23:27,960 --> 00:23:31,080 of whittle down like what could be wrong? 431 00:23:31,240 --> 00:23:35,920 What, what input could I give to, to my human to help us think 432 00:23:35,920 --> 00:23:39,840 this through? So it's yeah, it is. 433 00:23:39,840 --> 00:23:43,480 It's just really interesting the the other side of that. 434 00:23:43,480 --> 00:23:48,680 The other side of that is, by the way, that any sort of 435 00:23:48,680 --> 00:23:52,440 meaningfully complex component will have a data sheet that will 436 00:23:52,520 --> 00:23:56,040 overwhelm the alarm. So very frequently I built up 437 00:23:56,040 --> 00:24:01,800 sort of helper helper files, summaries of data sheets from a 438 00:24:01,800 --> 00:24:04,120 particular perspective, you know, tell me anything I need to 439 00:24:04,120 --> 00:24:08,680 know about the IO pins, for instance, and you know what, 440 00:24:08,680 --> 00:24:11,000 what voltages they supply, let's say. 441 00:24:11,960 --> 00:24:14,080 So that's sort of that. I have that sort of shelved. 442 00:24:14,960 --> 00:24:17,160 I can stick this particular aspect in. 443 00:24:17,680 --> 00:24:20,320 So you would go through ahead of time and you would go and, and 444 00:24:20,320 --> 00:24:23,000 this is where I think a lot of people, I don't know, I, when I 445 00:24:23,000 --> 00:24:25,760 first started, it was like, OK, I have my 1 interaction with my 446 00:24:25,760 --> 00:24:29,080 AI agent. No, no, you can have 10/15/20 of 447 00:24:29,080 --> 00:24:30,680 them open at the same time like cares. 448 00:24:30,960 --> 00:24:34,200 So if you were to take another agent and say, all right, open 449 00:24:34,200 --> 00:24:38,680 the status sheet and figure out, pick out everything about the IO 450 00:24:38,680 --> 00:24:42,080 pens and create another markdown file over here that has the 451 00:24:42,080 --> 00:24:44,880 information that's about the IO pens for this, for this device. 452 00:24:45,200 --> 00:24:47,840 Because you will find that you can't just grab a couple pages. 453 00:24:47,840 --> 00:24:50,160 Like you can't just go and like, oh, let me trim this. 454 00:24:50,160 --> 00:24:52,400 Let me take this PDF and like cut these 4 pages out. 455 00:24:52,400 --> 00:24:54,080 No, no, no, there's information here. 456 00:24:54,080 --> 00:24:55,640 There's information here. There's information in the 457 00:24:55,640 --> 00:24:57,000 introduction. There's other section 458 00:24:57,000 --> 00:24:58,560 completely. There's an appendix that's got 459 00:24:58,760 --> 00:25:02,440 like, so the data is actually spread across a lot of area. 460 00:25:02,680 --> 00:25:06,240 So to take an lol, parse that data sheet and then no, no, 461 00:25:06,280 --> 00:25:10,200 here's what I care about across all of these different sections. 462 00:25:10,200 --> 00:25:12,600 This is from this standpoint that's really interesting. 463 00:25:12,600 --> 00:25:15,680 And then being able to use that to then feed that to another 464 00:25:16,200 --> 00:25:19,720 agent to go and and and do this, that's doing the actual task. 465 00:25:20,680 --> 00:25:23,200 Let me point out that I tend not to do that ahead of time. 466 00:25:23,240 --> 00:25:26,480 It's except maybe for things that I I'm sure that I'm going 467 00:25:26,480 --> 00:25:28,720 to need eventually. But you can do it in line. 468 00:25:28,720 --> 00:25:32,160 You can, you can stop because we now open a new agent and do that 469 00:25:32,160 --> 00:25:34,640 and then feed that in. You don't have to use the same 470 00:25:34,640 --> 00:25:36,880 agent that you're currently working with the the. 471 00:25:37,320 --> 00:25:38,560 In fact, you shouldn't, as far as I'm sure. 472 00:25:38,560 --> 00:25:40,880 You should not, yeah, because you're including your contacts 473 00:25:40,880 --> 00:25:41,720 window. That's the whole thing. 474 00:25:41,720 --> 00:25:43,280 You talk about managing your contacts window like. 475 00:25:43,280 --> 00:25:46,040 No, no, this task is completely separate and all I need is the 476 00:25:46,040 --> 00:25:47,840 output from this task. So let me go and start a 477 00:25:47,840 --> 00:25:49,320 different one, yeah? Exactly. 478 00:25:49,320 --> 00:25:51,080 But but this is this is how I tend to work. 479 00:25:51,080 --> 00:25:54,120 Like at the moment I'm analyzing a very big code base. 480 00:25:55,000 --> 00:26:00,080 I need to frequently extract signal definitions for, you 481 00:26:00,080 --> 00:26:03,560 know, for, for bus messages. And they're all in the code 482 00:26:03,560 --> 00:26:06,360 there somewhere. But I need to go do some code 483 00:26:06,680 --> 00:26:09,800 archaeology. And I, I have a bunch of skills 484 00:26:09,800 --> 00:26:13,040 now that sort of systematically go through and make sure that 485 00:26:13,040 --> 00:26:16,640 nothing gets missed and that extract, for example, a new 486 00:26:16,640 --> 00:26:20,960 signal definition and place that in an agreed upon format in an 487 00:26:20,960 --> 00:26:24,880 agreed upon place. Now I have it ready to work. 488 00:26:25,440 --> 00:26:28,200 Uh, you know, it has all sorts of fanciness like it it, it 489 00:26:28,200 --> 00:26:32,120 always has Providence information like I, it's not 490 00:26:32,120 --> 00:26:34,520 only a description. It also says, and I got it from 491 00:26:34,520 --> 00:26:35,720 this file over there, right, right. 492 00:26:35,960 --> 00:26:41,000 So that I have a way of proving that it's not just hallucinated. 493 00:26:41,000 --> 00:26:43,760 I I can actually cross reference it and say, well, yeah, this is 494 00:26:43,960 --> 00:26:46,040 this is actually information, but it's actually in that file. 495 00:26:46,400 --> 00:26:47,720 Yeah. And this is this is what we 496 00:26:47,720 --> 00:26:50,640 talked about with Michael a few weeks ago, like about parsing 497 00:26:50,640 --> 00:26:54,480 these these spec files and and producing A consistent format on 498 00:26:54,480 --> 00:26:56,200 the other side. So like, OK, this is how I'm 499 00:26:56,200 --> 00:26:58,560 going to parse this. It can be used by something 500 00:26:58,560 --> 00:26:59,760 else. Exactly. 501 00:26:59,760 --> 00:27:03,040 And so this is one of those tricks that also makes debugging 502 00:27:03,040 --> 00:27:07,160 much, much, much more efficient because you've got this sort of 503 00:27:07,160 --> 00:27:11,240 distilled context in here that really helps the LLM to figure 504 00:27:11,240 --> 00:27:13,800 out, OK, what, what do I need to pay attention to? 505 00:27:14,040 --> 00:27:16,120 What's important? And then you can, and then you 506 00:27:16,120 --> 00:27:19,440 can iterate very quickly and, and try a couple of different 507 00:27:19,440 --> 00:27:23,720 experiments and use the human sort of stand back a little and 508 00:27:23,720 --> 00:27:26,480 watch whether it's still going in the right direction, whether 509 00:27:26,480 --> 00:27:28,960 it's starting to go in circles, whether it just lost the plot 510 00:27:28,960 --> 00:27:30,920 and it's doing, you know, whatever stupid thing. 511 00:27:31,520 --> 00:27:36,160 And it's just much, much quicker and much less exhausting, both 512 00:27:36,280 --> 00:27:38,360 mentally and sort of emotionally. 513 00:27:38,720 --> 00:27:41,560 Yep, Yep. Than than the other should we? 514 00:27:42,240 --> 00:27:46,920 Well, in the I, I, I think, I think a main point of our entire 515 00:27:46,920 --> 00:27:52,480 podcast is we're not replacing brains. 516 00:27:52,480 --> 00:27:55,760 We're augmenting. We're giving you more ability 517 00:27:55,760 --> 00:27:59,480 to, to do what you want to do without having to expand your 518 00:27:59,480 --> 00:28:02,640 own contact, without having to spend your own emotional and 519 00:28:02,640 --> 00:28:04,600 mental state in order to accomplish the task you're 520 00:28:04,600 --> 00:28:06,280 trying to do. You're able to accomplish more 521 00:28:06,280 --> 00:28:09,920 because of this, because of your interactions with, with an AI 522 00:28:09,920 --> 00:28:12,840 doing this kind of work. We're not replacing engineering 523 00:28:13,080 --> 00:28:14,640 effort. We're changing the way you 524 00:28:14,840 --> 00:28:18,440 engineer because the expertise is still absolutely necessary. 525 00:28:18,440 --> 00:28:22,120 So the the solution in my situation, after taking all 526 00:28:22,120 --> 00:28:24,200 these pictures and going back and forth with this, doing a 527 00:28:24,200 --> 00:28:27,880 little bit of manual Googling around myself and found out that 528 00:28:27,880 --> 00:28:30,520 the Elegoo actually has trouble with the device that it's 529 00:28:30,520 --> 00:28:35,280 shipped with the starter kit. This this specific device has 530 00:28:35,280 --> 00:28:39,720 trouble with this specific board and it's because of the five 531 00:28:39,720 --> 00:28:43,200 bolt digital pins. And so the solution that people 532 00:28:43,200 --> 00:28:47,440 say is like either get a different, different connection, 533 00:28:47,480 --> 00:28:50,080 get, get, get a completely different sensor to do the same 534 00:28:50,080 --> 00:28:53,720 thing, or you've got to step down the power on all the 535 00:28:53,720 --> 00:28:56,680 digital pins as well. And I was like, but it, it was 536 00:28:56,680 --> 00:29:00,040 only for this specific LL goo when you looked it up on the I 537 00:29:00,040 --> 00:29:03,320 so many things, but I found in one form and like the LLM is 538 00:29:03,320 --> 00:29:06,680 boiling the Internet in order to to give you information. 539 00:29:06,680 --> 00:29:10,000 So it's really tough to get specific information out of the 540 00:29:10,000 --> 00:29:12,600 LLM, but it'll give you generalized information. 541 00:29:12,600 --> 00:29:16,320 So to get the very specific thing that you need, it's like, 542 00:29:17,080 --> 00:29:18,080 I don't know that I ever would have. 543 00:29:18,080 --> 00:29:21,200 That is such a nasty. This is such a nasty failure 544 00:29:21,200 --> 00:29:24,840 mode of LLMS, by the way. If they've for, for, you know, 545 00:29:24,840 --> 00:29:27,480 they, they know a lot about Arduinos, for instance, like in 546 00:29:27,480 --> 00:29:28,600 your example. Yeah. 547 00:29:28,600 --> 00:29:30,800 And if you've got one that that's sort of deviates just a 548 00:29:30,800 --> 00:29:34,080 little bit, just a little bit, they, yeah, they, they will 549 00:29:34,480 --> 00:29:38,680 always go back into that same rut, even though you tell them 550 00:29:38,680 --> 00:29:40,400 no, no, no, you know, this one is different. 551 00:29:40,680 --> 00:29:43,520 It has 5 Volt pins, not 3.3 Volt pins or something. 552 00:29:44,080 --> 00:29:47,160 It was like, Oh yeah, yeah, 5 Volt pins, 5 Volt pins and and 553 00:29:47,160 --> 00:29:51,880 then sort of ignore that and still think of 3.3 Volt pins 554 00:29:51,880 --> 00:29:55,480 because it's so hard for the LLM to resist this sort of pull off 555 00:29:56,000 --> 00:29:59,400 of how should I put this of of the majority Yeah, of the 556 00:29:59,400 --> 00:30:01,120 signal, right. It's always pulling the 557 00:30:01,120 --> 00:30:02,720 audience. If you remember, have you ever 558 00:30:02,720 --> 00:30:04,200 watched Who Wants to Be a Millionaire? 559 00:30:04,200 --> 00:30:06,440 Like the LLM is always pulling the audience. 560 00:30:06,440 --> 00:30:11,880 Like, yeah, exactly. And. 561 00:30:11,920 --> 00:30:13,800 Was them of the mass? I believe that's called? 562 00:30:13,800 --> 00:30:19,200 Is that that? Yeah, but I, I see this in real, 563 00:30:19,360 --> 00:30:25,320 real life where I was trying to get a, a simulator to run and 564 00:30:25,480 --> 00:30:28,800 it, there was like a version 1 and version 2 or version 2 and 565 00:30:28,800 --> 00:30:31,800 version three or something. And the version, the, the older 566 00:30:31,800 --> 00:30:35,080 version just had a lot more mindshare because it'd been, 567 00:30:35,120 --> 00:30:36,840 it's, it had been around so long. 568 00:30:37,320 --> 00:30:43,000 And so the LLM keep putting in API calls to an API that wasn't 569 00:30:43,000 --> 00:30:45,920 present anymore. And then it tried to compile and 570 00:30:45,920 --> 00:30:48,600 said, oh, that's right, this is the V2 API. 571 00:30:48,600 --> 00:30:52,240 You need to use the V3 API and replace it with the correct call 572 00:30:52,240 --> 00:30:56,600 that it also knew, only to make the same mistake yet again. 573 00:30:56,800 --> 00:30:58,840 So it was really interesting to observe it. 574 00:30:59,200 --> 00:31:01,760 And in this case, luckily, it was very harmless, you know, 575 00:31:01,800 --> 00:31:06,600 because it was such an obvious mistake that you know, the code 576 00:31:06,600 --> 00:31:09,400 wouldn't compile, so it couldn't, it couldn't possibly 577 00:31:09,400 --> 00:31:12,960 sneak by you. But if this were more subtle, 578 00:31:13,560 --> 00:31:16,720 you know it, it would be so nasty, it would it would, you 579 00:31:16,720 --> 00:31:22,120 know, despite all of its try really hard to please you, it 580 00:31:22,120 --> 00:31:25,400 would put in the wrong kind of code, the wrong kind of behavior 581 00:31:25,760 --> 00:31:27,520 because it just can't hide itself. 582 00:31:28,040 --> 00:31:30,200 Oh my God, what, what are we to do? 583 00:31:30,920 --> 00:31:33,240 What are we to do that? And that's, I, I think a lot of 584 00:31:33,240 --> 00:31:37,440 folks out there like that very statement, that very sentiment 585 00:31:37,440 --> 00:31:40,240 right there is, is preventing a lot of people from even trying 586 00:31:40,240 --> 00:31:43,440 this out because like, well, if it's not 100%, if it's not 100%, 587 00:31:43,440 --> 00:31:45,880 I can't trust it at all. No, you can't. 588 00:31:46,560 --> 00:31:52,480 I mean, that's beautiful, right? But but The thing is, I can't 589 00:31:52,480 --> 00:31:55,440 trust myself 100% either, you know? 590 00:31:56,680 --> 00:31:59,000 What am I to do? How do these how do these 591 00:31:59,040 --> 00:32:02,000 conversations about OM turn into like metaphysical conversations 592 00:32:02,120 --> 00:32:05,560 like. No, but, but the point is, yeah, 593 00:32:05,680 --> 00:32:08,680 I I think that the, the point that is sort of just below the 594 00:32:08,680 --> 00:32:13,680 surface here is that you cannot use an LLM as a replacement for 595 00:32:13,680 --> 00:32:15,360 your brain or for your experience. 596 00:32:15,640 --> 00:32:19,360 But you can use it as a force multiplier like you can have, 597 00:32:19,840 --> 00:32:25,360 you can show it a picture of your circuit and it can provide 598 00:32:25,360 --> 00:32:28,240 you hypotheses about what might be wrong. 599 00:32:28,280 --> 00:32:32,520 And it's on you to verify that and to ensure that nothing 600 00:32:32,520 --> 00:32:35,320 stupid happens because it, it proposed something that was just 601 00:32:35,320 --> 00:32:40,480 like just egregiously wrong. But it gives you a new impulse 602 00:32:41,000 --> 00:32:44,440 and you can use it to pursue certain hypotheses like, OK, 603 00:32:44,440 --> 00:32:45,920 fine. I, I think maybe the problem is 604 00:32:45,920 --> 00:32:47,960 in here. Let's put a couple of printer 605 00:32:47,960 --> 00:32:51,320 statements and then see what the actual, you know, execution flow 606 00:32:51,320 --> 00:32:54,120 is or something, and go from there. 607 00:32:54,120 --> 00:32:57,600 So it is a force multiplier and the force multiplier is greater 608 00:32:57,600 --> 00:33:00,520 and greater the more experienced you are. 609 00:33:01,200 --> 00:33:04,520 This is something that I talked about with Jacob Beningo a while 610 00:33:04,520 --> 00:33:07,800 ago where he said that his experience and the experience 611 00:33:07,840 --> 00:33:12,440 of, of other senior people was that they are really, you know, 612 00:33:12,480 --> 00:33:16,000 a couple of times more efficient using an LLM, whereas a junior 613 00:33:16,520 --> 00:33:20,000 is, you know, a couple of percentage points better. 614 00:33:20,400 --> 00:33:24,680 So it's, it's like not for nothing, but they, they don't 615 00:33:24,680 --> 00:33:27,880 have the, the experience to really guide the LLM and, and 616 00:33:28,480 --> 00:33:30,960 not make particular mistakes that the LLM wants to make. 617 00:33:30,960 --> 00:33:35,200 Like just today I had the LLM, you know, we, we talked about my 618 00:33:35,200 --> 00:33:39,560 TDD workflow just in the last episode and I had to create a 619 00:33:39,560 --> 00:33:45,160 couple of test ideas and it was fantastic because for one, the 620 00:33:45,160 --> 00:33:48,320 test ideas exposed gaps in my own specification. 621 00:33:49,080 --> 00:33:51,800 Yeah. And yeah. 622 00:33:52,040 --> 00:33:55,480 And for another, it allowed me, you know, to, to sort of harden 623 00:33:55,480 --> 00:33:59,200 that and, and really hammer out a, a correct system. 624 00:33:59,640 --> 00:34:02,280 But also it showed me OK, the, the LLM could not have been 625 00:34:02,280 --> 00:34:05,160 trusted to do this by itself because it just, it missed a 626 00:34:05,160 --> 00:34:08,000 couple of honestly fairly obvious things. 627 00:34:08,000 --> 00:34:12,400 And, and to be honest, I missed them as well, at least 628 00:34:12,400 --> 00:34:14,679 initially. But then sort of going over the 629 00:34:14,679 --> 00:34:18,280 proposals that the, that the AI made, I was thinking, oh, you 630 00:34:18,280 --> 00:34:20,920 know, there's a gap in here. I can see it. 631 00:34:21,639 --> 00:34:26,320 And out of like 14 test cases, I think I, I commented on 8 of 632 00:34:26,320 --> 00:34:29,159 them or something. And now it's good and now it's 633 00:34:29,159 --> 00:34:32,120 valuable. But it, it, it required somebody 634 00:34:32,120 --> 00:34:36,800 with, you know, taste almost about how to design a system, 635 00:34:37,560 --> 00:34:40,600 right. And as, yeah, like, like with 636 00:34:40,600 --> 00:34:42,679 many of those architecture question is not really, you 637 00:34:42,679 --> 00:34:44,239 could say it's correct or incorrect. 638 00:34:44,239 --> 00:34:48,679 It's just OK, I, I can, I can see the obvious trade-offs here. 639 00:34:48,679 --> 00:34:50,480 And they are not going the way I want them to. 640 00:34:50,760 --> 00:34:56,320 Let me make them differently. So again, LLMS fantastic force 641 00:34:56,320 --> 00:34:58,680 multiplier. Also in debugging, they will 642 00:34:58,920 --> 00:35:02,880 maybe not in all cases find the bug by themselves, but they will 643 00:35:02,880 --> 00:35:07,360 make it so much quicker for you to get to the to the actual 644 00:35:07,360 --> 00:35:10,360 crucial place there is where it's hiding. 645 00:35:11,000 --> 00:35:13,800 Now let's go and fix it. Yep, Yep. 646 00:35:14,120 --> 00:35:17,920 Well, and it's ability to, to parse through a large code base 647 00:35:17,920 --> 00:35:20,240 too. You know, this is where the, 648 00:35:20,320 --> 00:35:22,080 that context management's important too. 649 00:35:22,080 --> 00:35:24,840 If you were you're, you don't want to develop and debug in the 650 00:35:24,840 --> 00:35:27,160 same thing, you got to set up another thing to go and debug so 651 00:35:27,160 --> 00:35:29,360 that it can start with a fresh context. 652 00:35:29,360 --> 00:35:31,160 But then you need to give it enough like, OK, this is where 653 00:35:31,160 --> 00:35:33,920 I'm starting. And then kind of have it it can 654 00:35:33,920 --> 00:35:36,640 go back and actually go through the the source code faster than 655 00:35:36,640 --> 00:35:40,200 you can to understand how all the the data is getting through 656 00:35:40,200 --> 00:35:41,800 the system and getting to the wrong spot. 657 00:35:42,320 --> 00:35:46,160 Yeah, that that that is so, so useful also in in testing like 658 00:35:46,480 --> 00:35:50,000 you can just have it go through an entire log file and say, 659 00:35:50,000 --> 00:35:52,920 well, is there something out of the ordinary? 660 00:35:53,480 --> 00:35:58,920 And I like to still have it flanked by by regular scripts 661 00:35:58,920 --> 00:36:01,000 that go, you know, for keywords or something. 662 00:36:01,240 --> 00:36:04,480 We're now OK, if, if this happens, you know, if I if I 663 00:36:04,480 --> 00:36:07,080 spot this keyword, then it's a signal that that something 664 00:36:07,400 --> 00:36:10,680 important has happened. And I will have the LLM write 665 00:36:10,680 --> 00:36:13,480 such scripts for me sort of almost one off. 666 00:36:14,240 --> 00:36:17,120 But you can also just say, OK, look through this file. 667 00:36:17,120 --> 00:36:18,840 Does something look funny to you? 668 00:36:19,120 --> 00:36:22,440 Yeah. And and that is so powerful that 669 00:36:22,440 --> 00:36:25,080 that LMS have this concept of looks funny. 670 00:36:25,600 --> 00:36:32,080 Right. Extremely, extremely powerful 671 00:36:32,080 --> 00:36:33,920 and I've used that to to great advantage. 672 00:36:33,920 --> 00:36:37,440 For example, last year when, when I had a rather insidious 673 00:36:38,000 --> 00:36:41,120 data corruption bug in in, in a set of his file system. 674 00:36:42,360 --> 00:36:45,800 And and like together with the LLM, we would, we would churn 675 00:36:45,800 --> 00:36:50,240 through the different options. And yes, I could have looked up 676 00:36:50,240 --> 00:36:54,280 all of the different CFS and that pool and ZET debug 677 00:36:54,280 --> 00:36:57,880 commands, but the AI already knew them. 678 00:36:58,000 --> 00:37:02,960 And so we were a lot faster and we could interpret the the blog 679 00:37:02,960 --> 00:37:06,160 messages a lot faster. And by the way, the LLM would 680 00:37:06,160 --> 00:37:10,000 still have destroyed my file system a couple of times if I 681 00:37:10,000 --> 00:37:13,720 had not looked through every command that it was proposing, 682 00:37:15,320 --> 00:37:19,200 so yeah. I have, I think you, you touched 683 00:37:19,200 --> 00:37:21,520 on something and we'll have to, we'll have to bring that up in 684 00:37:21,520 --> 00:37:24,640 another episode, but you touched on something about combining 685 00:37:24,640 --> 00:37:26,400 determinism with non determinism. 686 00:37:26,400 --> 00:37:29,680 So like adding scripts and running those scripts that are 687 00:37:29,680 --> 00:37:33,280 deterministic and will give you consistent results every single 688 00:37:33,280 --> 00:37:34,800 time because they are deterministic. 689 00:37:35,320 --> 00:37:38,520 And then having the LOM run those like and and you know, you 690 00:37:38,520 --> 00:37:40,160 just pointed out like, I didn't write the script. 691 00:37:40,200 --> 00:37:43,880 I took another agent, open it up, Ashley wrote the script of 692 00:37:43,880 --> 00:37:46,000 that agent. Then I'm using that script later 693 00:37:46,240 --> 00:37:49,400 to do specific tasks that give me that deterministic thing 694 00:37:49,400 --> 00:37:52,240 because there are things that don't need to be non 695 00:37:52,240 --> 00:37:55,280 deterministic. Like that's absolutely true. 696 00:37:55,280 --> 00:37:59,040 I do not need it trying to figure out how to, you know, 697 00:37:59,040 --> 00:38:01,160 compile the thing. Like I can write a script for 698 00:38:01,160 --> 00:38:03,440 that old, compile it the same way every single time. 699 00:38:03,720 --> 00:38:06,640 I don't need to go through and like have it go try to, you 700 00:38:06,640 --> 00:38:09,000 know, non deterministically figure out the C mate command 701 00:38:09,000 --> 00:38:10,920 that I'm trying to run. Like, no, just put a script in 702 00:38:10,920 --> 00:38:14,840 place, have it deterministically do that, you know, similar with 703 00:38:14,840 --> 00:38:17,760 this, some of the validation and the testing stuff like no, this 704 00:38:17,760 --> 00:38:19,680 is the like, that's why the test need to pass. 705 00:38:20,280 --> 00:38:23,000 So we, we have that deterministic testing script 706 00:38:23,000 --> 00:38:24,600 that'll go through. Now you don't need to write it. 707 00:38:24,600 --> 00:38:26,720 Go through with another agent, create that testing script. 708 00:38:26,920 --> 00:38:29,960 Debugging's the same way. I don't need you to figure out 709 00:38:29,960 --> 00:38:31,600 all these different things every single time. 710 00:38:31,600 --> 00:38:35,160 Use, use this specific command, use this specific script to 711 00:38:35,160 --> 00:38:38,200 start the debugging session, but then non deterministically parse 712 00:38:38,200 --> 00:38:40,240 through the output and figure it out that way. 713 00:38:40,240 --> 00:38:43,920 So like there's this balance and trying to, and I don't, there's 714 00:38:43,920 --> 00:38:47,160 no, this is, this is building expertise, just like the, the 715 00:38:47,160 --> 00:38:51,200 thing that people are worried about with, with, with juniors 716 00:38:51,200 --> 00:38:54,200 or newer people that are, that are coming to this and their 717 00:38:54,360 --> 00:38:58,560 dependence on LMS, like you have to build up the expertise still. 718 00:38:58,560 --> 00:39:02,040 And so there's, there's building up the expertise traditionally 719 00:39:02,040 --> 00:39:03,960 that we've talked about. And some of us are, you know, 720 00:39:03,960 --> 00:39:07,760 very reluctant to kind of to step into these AI coding agents 721 00:39:07,960 --> 00:39:11,200 and getting into this space because we've built up this, 722 00:39:11,600 --> 00:39:14,400 this, this institution of knowledge that we have. 723 00:39:15,040 --> 00:39:18,400 And we're so, and there is a lot of fear that the, the next 724 00:39:18,400 --> 00:39:22,600 generation won't have built that up because they'll be relying on 725 00:39:22,600 --> 00:39:25,560 LLMS. But if you're actually being 726 00:39:25,560 --> 00:39:28,600 effective with the LLMS, you have generated a different set 727 00:39:28,600 --> 00:39:33,760 of skills that are both using the LLM and understanding enough 728 00:39:33,760 --> 00:39:36,800 about the hardware. And I, I think that's, I think 729 00:39:36,800 --> 00:39:38,160 that's where what's going to happen. 730 00:39:38,520 --> 00:39:41,280 But I think the fear is that we won't actually generate that 731 00:39:41,280 --> 00:39:43,760 other skill set. We'll only generate the skill 732 00:39:43,760 --> 00:39:45,320 set of learning how to interact the LLM. 733 00:39:45,320 --> 00:39:47,840 And that's just you it. It's not possible. 734 00:39:47,840 --> 00:39:50,080 Like you can't actually solve these problems and and get it 735 00:39:50,080 --> 00:39:51,640 done. You have to generate both. 736 00:39:52,360 --> 00:39:53,600 Yeah. And and I think it's going to 737 00:39:53,600 --> 00:39:55,760 happen naturally. Like you can get to a certain 738 00:39:55,760 --> 00:39:59,840 point with LMS and then and then you're going to get into trouble 739 00:39:59,840 --> 00:40:03,320 and then, you know, it just takes very old fashioned 740 00:40:03,320 --> 00:40:05,920 determination and, and sort of struggling through it. 741 00:40:06,480 --> 00:40:08,120 Yeah. Supported by LMS, fine, 742 00:40:08,280 --> 00:40:13,080 whatever, but so I I don't think humans are are going away in 743 00:40:13,080 --> 00:40:15,160 that sense at all. Nope. 744 00:40:16,480 --> 00:40:19,080 Nope. Yeah, but but to to sort of 745 00:40:19,080 --> 00:40:24,640 summarize, so how, how do you actually use AI for debugging, 746 00:40:24,640 --> 00:40:26,360 especially in the embedded systems context? 747 00:40:26,480 --> 00:40:30,520 So close the loop, right? Get the LLM to write print of 748 00:40:30,520 --> 00:40:34,360 statements for instance, and get the LLM to read the UR output 749 00:40:34,360 --> 00:40:39,520 for you and read, you know, read it's own, read it's own outputs. 750 00:40:40,280 --> 00:40:43,600 You will curate the the context for it. 751 00:40:44,200 --> 00:40:50,000 You will watch over it to not go off the rails, not lose sight of 752 00:40:50,000 --> 00:40:52,280 where it's going and not go in circles. 753 00:40:52,920 --> 00:40:57,600 Maybe open another window and create supporting materials for 754 00:40:57,600 --> 00:41:03,840 it, summarizations of of data sheets or, you know, web 755 00:41:03,840 --> 00:41:07,160 research or something like that, or deterministic scripts that 756 00:41:07,280 --> 00:41:11,680 supported in some way. And, and that way you, you know, 757 00:41:11,840 --> 00:41:17,080 you control the, the overarching big lines and it, it does the, 758 00:41:17,480 --> 00:41:21,040 the sort of nitty gritty work. And then and that is good old 759 00:41:21,080 --> 00:41:23,840 iteration like like always, isn't it? 760 00:41:24,480 --> 00:41:28,640 Yep, and then and then stepping out and like, I don't know, 761 00:41:28,920 --> 00:41:31,640 using your webcam, using your phone and being able to take a 762 00:41:31,640 --> 00:41:34,560 photo and say like, hey, this is this is the way it looks. 763 00:41:34,560 --> 00:41:37,200 I mean, it's surprising that it's able to, to kind of 764 00:41:37,200 --> 00:41:40,560 identify that and, and, and help debug it in that way too. 765 00:41:40,560 --> 00:41:43,400 So I think that's, I think a lot of people are missing on that 766 00:41:43,400 --> 00:41:46,360 one too, where it's like, oh, no, we can actually go through 767 00:41:46,360 --> 00:41:48,480 this picture and understand what's happening on this 768 00:41:48,480 --> 00:41:53,200 picture, which it it literally like I had, it wasn't a ribbon 769 00:41:53,200 --> 00:41:55,280 cable, but I left it a bunch of cables that were there. 770 00:41:55,280 --> 00:41:56,840 I was just like, these are all next to each other. 771 00:41:56,840 --> 00:41:58,200 That's fine. I'll just pull them apart at the 772 00:41:58,200 --> 00:41:59,920 bottom. It's like, oh, you have a ribbon 773 00:41:59,920 --> 00:42:01,000 cable plugged in. That's wrong. 774 00:42:01,000 --> 00:42:01,920 You shouldn't do that. I'm like. 775 00:42:05,200 --> 00:42:10,320 Like tell me what to do. I don't, but like, OK, I mean, I 776 00:42:10,600 --> 00:42:12,280 I didn't feel like pulling them apart because then I'm like, 777 00:42:12,440 --> 00:42:13,760 then I put it back in the box later. 778 00:42:13,760 --> 00:42:15,600 I'm going to have to like put and I just wanted to leave them. 779 00:42:15,600 --> 00:42:18,120 It's OK. I don't want to explain this to 780 00:42:18,120 --> 00:42:24,160 you, man, but it was it notices like it's your ribbon cable. 781 00:42:24,160 --> 00:42:28,400 And I'm like, wow, wow. All right, good, Good point. 782 00:42:28,440 --> 00:42:31,160 One by the way, one thing I haven't tried that but I I 783 00:42:31,160 --> 00:42:34,960 suspect would work very well as well would be to show it 784 00:42:35,440 --> 00:42:39,640 screenshots of like oscilloscope pictures and say well this is 785 00:42:39,640 --> 00:42:44,200 the timing that I observed. I've I have had I've I had 786 00:42:44,200 --> 00:42:46,880 somebody comment on one of my YouTube videos of the day and 787 00:42:46,880 --> 00:42:49,800 they talked about they have the LOM reading the oscilloscope. 788 00:42:50,320 --> 00:42:53,080 Yeah, and I, I'm sure that works really well because I think it's 789 00:42:53,080 --> 00:42:56,040 much better than humans and sort of reasoning through timing 790 00:42:56,040 --> 00:42:59,000 changes OK, 'cause this and waste time there and then oh an 791 00:42:59,000 --> 00:43:01,960 interrupt happens, whatever so useful. 792 00:43:01,960 --> 00:43:05,760 And even if it gets it wrong, like it, it will sort of have 793 00:43:05,760 --> 00:43:08,240 pointed to all the important places. 794 00:43:08,240 --> 00:43:11,080 And as you sort of follow its reasoning, you'll say, oh, well, 795 00:43:11,440 --> 00:43:14,400 you know, you misunderstood. This doesn't matter. 796 00:43:14,480 --> 00:43:15,760 It pointed you at the right places. 797 00:43:15,760 --> 00:43:18,640 You didn't have to go rummage to record and find them for 798 00:43:18,640 --> 00:43:21,520 yourself. Yeah, yeah, gosh, didn't even 799 00:43:21,640 --> 00:43:30,080 get to our soloscopes. All right, lovely chatting with 800 00:43:30,080 --> 00:43:31,800 you today, Sir. This has been another episode of 801 00:43:31,800 --> 00:43:33,920 the Embedded AI podcast. I've been Ryan. 802 00:43:34,080 --> 00:43:36,920 I'm continuing to be Ryan Tarbic after this, but I have been and 803 00:43:36,920 --> 00:43:42,760 will continue to be Ryan Tarbic. And likewise, I am and will 804 00:43:42,760 --> 00:43:46,280 continue to be Luca Jenny. We'll catch you guys next time. 805 00:43:47,600 --> 00:43:53,280 See ya. Hi, Luca here. 806 00:43:53,760 --> 00:43:56,680 I have a quick announcement. I've just launched one more new 807 00:43:56,680 --> 00:44:00,680 training platform, the Embedded AI Academy at Embedded AI dot 808 00:44:00,680 --> 00:44:02,760 Academy. It feels like up I've seen in 809 00:44:02,760 --> 00:44:06,040 the market training that approaches AI specifically from 810 00:44:06,040 --> 00:44:08,360 an embedded systems angle. If you're listening to this 811 00:44:08,360 --> 00:44:12,120 podcast, it should be right up your alley and you get 25% off 812 00:44:12,120 --> 00:44:15,480 any booking made through the end of 2025 if you use the code 813 00:44:15,800 --> 00:44:19,680 Embedded AI Podcast 25 all in capital letters. 814 00:44:20,080 --> 00:44:23,680 Again, the website is at Embedded AI Academy.