1 00:00:00,040 --> 00:00:03,080 One of the hot topics in AI coding these days is the Ralph 2 00:00:03,080 --> 00:00:05,160 Wiggum loop. This was made months ago by 3 00:00:05,160 --> 00:00:08,440 Jeffrey Huntley, popularized by Dexter Horthy, Ryan Carson, and 4 00:00:08,440 --> 00:00:10,040 many others. And if you've watched The 5 00:00:10,040 --> 00:00:12,880 Simpsons before, you know that Ralph Wiggum is simple yet 6 00:00:12,880 --> 00:00:14,560 determined. And that's a great way to sum up 7 00:00:14,560 --> 00:00:16,840 what the loop is. On episode 76 of 2 of Use, we're 8 00:00:16,840 --> 00:00:19,560 joined by Ryan Carson. He was with us on episode 49 9 00:00:19,560 --> 00:00:22,200 talking about how he built a one person company with the help of 10 00:00:22,200 --> 00:00:24,080 AI. Now, five months later, he's 11 00:00:24,080 --> 00:00:26,520 shipping faster than ever. So we're going to cover what the 12 00:00:26,520 --> 00:00:29,080 Ralph Wiggum loop is and how you can implement it yourself. 13 00:00:29,520 --> 00:00:32,479 Other loops that Ryan has found create real value for him and 14 00:00:32,479 --> 00:00:34,320 his business. And we're going to both share 15 00:00:34,320 --> 00:00:36,560 some of our experience building in this phase of the AI 16 00:00:36,560 --> 00:00:38,360 revolution. So please enjoy this 17 00:00:38,360 --> 00:00:40,680 conversation with Ryan Carson. Oh boy. 18 00:00:40,680 --> 00:00:43,760 So first of all, you have to know what The Simpsons is. 19 00:00:43,760 --> 00:00:47,880 So if you've if you watch The Simpsons, Ralph is the 20 00:00:47,880 --> 00:00:51,920 hilariously doofy character that doesn't really know anything. 21 00:00:52,160 --> 00:00:56,480 And so the idea of the Ralph Wiggum loop was created by Jeff 22 00:00:56,480 --> 00:00:59,720 Huntley. So Jeff Huntley worked at amp 23 00:00:59,720 --> 00:01:03,600 for a little while and he came up with this idea of creating a 24 00:01:04,120 --> 00:01:09,440 bash grip that just loops amp or cloud code or agent of choice 25 00:01:10,240 --> 00:01:13,840 and completes a list of tasks. And it sounds kind of dumb 26 00:01:13,840 --> 00:01:16,440 because it's so simple, but this is actually why it's it's 27 00:01:16,440 --> 00:01:19,720 beautiful. So here's how it works, right? 28 00:01:19,720 --> 00:01:24,560 So you have this idea for a feature or an app or a list of 29 00:01:24,560 --> 00:01:26,600 tasks that you want an agent to do. 30 00:01:27,520 --> 00:01:29,840 The idea is that it's supposed to be hard and complex, right? 31 00:01:29,840 --> 00:01:32,480 This, this is not anything that should be one shot. 32 00:01:32,600 --> 00:01:34,360 And then you create a list of tasks. 33 00:01:34,360 --> 00:01:36,400 You know, you can think of these as user stories. 34 00:01:37,520 --> 00:01:41,920 I use user stories with clear acceptance criteria that can be 35 00:01:41,920 --> 00:01:44,640 verified by an agent, right? You don't want any human in the 36 00:01:44,640 --> 00:01:46,760 loop for the acceptance criteria. 37 00:01:47,720 --> 00:01:51,600 And then you basically create a simple bash script or you have 38 00:01:51,960 --> 00:01:55,920 amp do it or clock code do it. I, I didn't write any bash 39 00:01:56,040 --> 00:02:01,280 myself. And it basically fires up the 40 00:02:01,280 --> 00:02:05,880 agent on the command line. So an amp it's amp dash X which 41 00:02:05,880 --> 00:02:09,720 is execute and then the string which is the prompt, right? 42 00:02:10,880 --> 00:02:12,640 And you can do this with clock code. 43 00:02:13,200 --> 00:02:18,240 And you, you have the loop fire up the agent with a prompt. 44 00:02:18,240 --> 00:02:22,440 And then what the prompt is, is basically the user story that 45 00:02:22,440 --> 00:02:26,800 you want it to complete. And then the idea is it works, 46 00:02:26,800 --> 00:02:30,040 works, works. And when it's done, it stops. 47 00:02:30,040 --> 00:02:34,040 And then the bash script loops again and picks up the next 48 00:02:34,040 --> 00:02:37,640 story, finishes it loops again, picks up the next story, 49 00:02:37,640 --> 00:02:39,680 finishes it, right. So it's literally just a loop 50 00:02:39,920 --> 00:02:43,720 completing a list of tasks. And again, people might think, 51 00:02:43,720 --> 00:02:46,200 well, I don't get it. Like what is what is the point? 52 00:02:46,720 --> 00:02:51,000 And the idea is every time you get a fresh context window and 53 00:02:51,000 --> 00:02:55,040 this is the the magic, right? The idea that every time the 54 00:02:55,040 --> 00:02:56,880 agent start, it has fresh context. 55 00:02:57,840 --> 00:03:01,640 So it's ready to go, it completes, it learns. 56 00:03:01,640 --> 00:03:05,280 So the way it learns is it uses the file system to store 57 00:03:05,400 --> 00:03:09,080 memories or, or things that it should remember for the next 58 00:03:09,080 --> 00:03:11,440 loop. I had this idea of, OK, you have 59 00:03:11,440 --> 00:03:14,960 long term memory, which is usually your agents dot MD 60 00:03:14,960 --> 00:03:17,520 files. This, these are things that you 61 00:03:17,520 --> 00:03:21,080 need the agent to remember all the time, no matter when it 62 00:03:21,080 --> 00:03:22,960 runs. But then you have this kind of 63 00:03:22,960 --> 00:03:25,480 short term memory of like, OK, I'm trying to build out this 64 00:03:25,480 --> 00:03:30,320 specific feature and I'm running into some gotchas or a couple 65 00:03:30,320 --> 00:03:33,080 things I've learned specifically about this feature. 66 00:03:33,640 --> 00:03:35,560 That's the type of stuff you don't want to start polluting 67 00:03:35,560 --> 00:03:39,040 your agents dot MD file with or or your system prompts or 68 00:03:39,040 --> 00:03:43,440 anything, but you want the Ralph loop next time it comes around 69 00:03:43,440 --> 00:03:47,120 to remember it. So in the system prompt for the 70 00:03:47,120 --> 00:03:53,000 RAF loop, you basically say pick a user story from this Jason 71 00:03:53,000 --> 00:03:55,160 file, right? And the Jason file is just a 72 00:03:55,160 --> 00:03:58,200 list of of tasks. You say pick one that's not 73 00:03:58,200 --> 00:04:04,840 complete and before you start it read progress dot TXT, right? 74 00:04:04,840 --> 00:04:06,560 And this is literally just a file. 75 00:04:06,640 --> 00:04:09,400 You know in your repo. It's not rocket science. 76 00:04:09,840 --> 00:04:12,880 And you and you say read it first and then execute the task. 77 00:04:13,280 --> 00:04:18,920 And when the task is done, mark the the Jason item as complete 78 00:04:19,160 --> 00:04:23,280 and then update progress dot TXT with anything you know that you 79 00:04:23,280 --> 00:04:27,920 kind of ran into that you that you should remember for other 80 00:04:28,440 --> 00:04:31,800 user stories in this feature. If there's anything long term 81 00:04:31,920 --> 00:04:35,280 that you think the agent should know, no matter what they're 82 00:04:35,280 --> 00:04:39,160 building in this repo, add it to agents dot MD and then quit. 83 00:04:39,800 --> 00:04:45,560 And you just loop. And, you know, I think, you 84 00:04:45,560 --> 00:04:47,680 know, a couple of folks like Matt Peacock have really 85 00:04:47,680 --> 00:04:50,160 summarized why this works so well. 86 00:04:50,720 --> 00:04:53,440 And it's because humans have figured out that this is the 87 00:04:53,440 --> 00:04:56,760 optimal way to work as a team, right? 88 00:04:56,760 --> 00:05:00,280 So you have sprints, you have, you know, a combine board with 89 00:05:00,280 --> 00:05:04,880 user stories. And what a dev would do is they 90 00:05:04,880 --> 00:05:07,720 would go up to the board and grab a user story because they 91 00:05:07,880 --> 00:05:09,520 they think, OK, I can tackle that. 92 00:05:09,680 --> 00:05:12,520 I can do this in isolation. They go back to the desk and 93 00:05:12,520 --> 00:05:15,720 they work on it, they complete it, they ship it and then they 94 00:05:15,880 --> 00:05:18,280 cross it off and they go grab another user story, right? 95 00:05:18,720 --> 00:05:21,320 This is exactly what a Ralph Wiggum loop is doing. 96 00:05:21,920 --> 00:05:25,400 It's just an agent instead of a human and then you can massively 97 00:05:25,400 --> 00:05:27,920 paralyze, right? So you could actually have, you 98 00:05:27,920 --> 00:05:31,640 know, 3 Ralph Wiggum loops running in parallel working on 99 00:05:31,640 --> 00:05:35,440 different features on the same repo. 100 00:05:36,120 --> 00:05:39,840 And personally, now I just use different clones on my local 101 00:05:39,840 --> 00:05:41,760 machine. I don't use get work trees. 102 00:05:41,760 --> 00:05:45,600 I find that confusing. So I just literally have three 103 00:05:45,600 --> 00:05:49,560 folders on my machine with that are three checkouts of the repo. 104 00:05:50,800 --> 00:05:54,360 And if I have, you know, Ralph Wiggum loop operating, I just 105 00:05:54,360 --> 00:05:57,680 have them operating on different checkouts and then, you know 106 00:05:57,720 --> 00:06:01,040 all, you know, pull rebase and then and then commit. 107 00:06:01,040 --> 00:06:05,400 So it's, it really works. I mean, I've, I've shipped 108 00:06:05,400 --> 00:06:11,000 probably 6-7 pretty big features and then when it's done, usually 109 00:06:11,000 --> 00:06:13,320 you, it'll take a couple threads to clean it up. 110 00:06:13,360 --> 00:06:18,200 Make sure it's 100% correct. You know, do a really good, you 111 00:06:18,200 --> 00:06:21,000 know, PR review on it, which you could do as part of the loop if 112 00:06:21,000 --> 00:06:24,640 you wanted and then you ship it. It's, it's nice hearing it from 113 00:06:24,640 --> 00:06:26,920 your perspective because when it first came across my radar, I 114 00:06:26,920 --> 00:06:29,280 thought, wow, this is a great way to burn a lot of tokens. 115 00:06:29,520 --> 00:06:32,560 And I mean with, with the brute force approach of just trying 116 00:06:32,560 --> 00:06:35,080 and trying, trying, you know, you can get there eventually. 117 00:06:35,320 --> 00:06:39,520 But I do agree that what makes us really unique and and have a 118 00:06:39,520 --> 00:06:41,400 lot of potential is the fact that you get a clear context 119 00:06:41,400 --> 00:06:42,280 window. You're not going to get the 120 00:06:42,280 --> 00:06:44,800 context route that comes from most long running tasks. 121 00:06:45,040 --> 00:06:48,080 You have static asset or may not necessarily static, but editable 122 00:06:48,080 --> 00:06:50,440 assets that it can reference on each loop they get updated. 123 00:06:50,440 --> 00:06:52,360 Yeah. And I mean in cursor that tends 124 00:06:52,360 --> 00:06:55,200 to be my flow anyways, where I have a a specific document 125 00:06:55,200 --> 00:06:57,400 marked down for the given initiative I'm working on. 126 00:06:57,640 --> 00:07:00,000 And after it does a certain task like update the doc and then the 127 00:07:00,000 --> 00:07:02,000 next thread is just please review the doc. 128 00:07:02,000 --> 00:07:04,280 Let's get started. And just doing that a lot more 129 00:07:04,280 --> 00:07:06,760 manually. Yep, I think it's interesting 130 00:07:06,760 --> 00:07:10,040 because it feels like we're all evolving in parallel, right? 131 00:07:10,360 --> 00:07:14,200 And a lot of us are discovering the same workflows in parallel 132 00:07:14,200 --> 00:07:16,280 and sort of ending up in the same place. 133 00:07:17,280 --> 00:07:21,600 And I remember it, you know, I launched a simple repo called AI 134 00:07:21,600 --> 00:07:24,920 Dev Tasks probably, gosh, probably a year and a half ago 135 00:07:24,920 --> 00:07:28,560 now. I was on Clairvaux show and I 136 00:07:28,560 --> 00:07:30,880 basically showed it and and it was so simple. 137 00:07:30,880 --> 00:07:32,880 It was dumb. It was like, OK, I'm going to 138 00:07:32,880 --> 00:07:34,760 start and I'm going to plan, right? 139 00:07:34,760 --> 00:07:37,240 I'm going to make a markdown file PRD. 140 00:07:37,240 --> 00:07:39,240 I'm going to take that PRD. I'm going to break it down into 141 00:07:39,240 --> 00:07:41,760 tasks. I'm going to create a task file, 142 00:07:42,200 --> 00:07:43,480 right? And then I'm going to have the 143 00:07:43,480 --> 00:07:45,960 agent loop over the task. And at the time, this was like 144 00:07:45,960 --> 00:07:51,200 cutting edge, but a lot of people were arriving at that 145 00:07:51,200 --> 00:07:54,720 same workflow because the agents were limited that way, right? 146 00:07:55,120 --> 00:07:59,240 And now we've moved on past it and and I've literally removed 147 00:07:59,240 --> 00:08:02,600 that third file from that repo that says you, you know, check 148 00:08:03,040 --> 00:08:06,200 you need a human in the loop after each iteration, which is 149 00:08:06,480 --> 00:08:07,760 crazy. You don't at all now. 150 00:08:08,000 --> 00:08:11,760 So I ripped that out. But now this whole PRD to task 151 00:08:11,760 --> 00:08:15,640 plan to execution phase, it turns out it's even better with 152 00:08:15,640 --> 00:08:18,800 the Ralph loop, right? And it is, it is token 153 00:08:18,800 --> 00:08:22,360 efficient. I think people, this is where I 154 00:08:22,360 --> 00:08:27,520 think Jeff got initially a bad name for Ralph because people 155 00:08:27,520 --> 00:08:30,680 thought it was just this token hungry, ridiculous loop. 156 00:08:32,280 --> 00:08:35,520 But it turns out that the agent is actually very token 157 00:08:35,520 --> 00:08:38,000 efficient. If you give it a clear user 158 00:08:38,000 --> 00:08:40,600 story, right, with clear acceptance criteria, it's 159 00:08:40,600 --> 00:08:43,640 actually probably more efficient than you being in the loop. 160 00:08:45,640 --> 00:08:51,480 So we're getting closer to what, to what feels, you know, like an 161 00:08:51,480 --> 00:08:56,880 automated team, but not quite. I'd like to actually go over if 162 00:08:56,920 --> 00:09:00,720 you're aware of any differences between what Jeff's original 163 00:09:00,720 --> 00:09:03,880 vision was and actually, sorry, quick note, A5 line bash file 164 00:09:03,880 --> 00:09:07,720 similar to how like successful theorems and theories come 165 00:09:07,720 --> 00:09:09,240 about. The the elegance is in the 166 00:09:09,240 --> 00:09:11,800 simplicity and A5 line thing that can do this type of thing 167 00:09:11,800 --> 00:09:14,240 is just so cool. But is there a big difference 168 00:09:14,240 --> 00:09:17,360 between when these harnesses apply it like quad code or AMP 169 00:09:17,440 --> 00:09:19,560 versus Jeff's initial vision for it? 170 00:09:19,880 --> 00:09:22,600 Yeah. So I I think the simple 171 00:09:22,600 --> 00:09:25,480 difference is that it it really should be executed in a bash 172 00:09:25,480 --> 00:09:28,680 loop. If you actually have the agent 173 00:09:28,680 --> 00:09:33,320 try to do it, it will often rot the context. 174 00:09:34,040 --> 00:09:38,760 So amp is a pretty cool feature that the team is working on that 175 00:09:38,760 --> 00:09:42,160 should be shipped soon. It's basically tasks with 176 00:09:42,160 --> 00:09:44,880 handoff. Handoff is already live, but 177 00:09:44,880 --> 00:09:49,760 that this idea of a a atomic unit of a task that could have 178 00:09:49,760 --> 00:09:54,520 parents or descendants and you basically spin up a thread and 179 00:09:54,520 --> 00:09:58,320 you you it's actually a skill. So I wrote a Ralph Wiggum skill 180 00:09:59,560 --> 00:10:02,600 and the idea is very similar to the bash loop. 181 00:10:02,600 --> 00:10:06,600 But you the agent says, OK here, you know, go get the the next 182 00:10:06,600 --> 00:10:12,840 task that is available, you know, read progress dot TXT and 183 00:10:12,840 --> 00:10:15,280 then execute it. And at the end, mark the task as 184 00:10:15,280 --> 00:10:19,760 complete update progress dot T and then use hand off to create 185 00:10:19,760 --> 00:10:23,760 a new thread. It's almost the same thing, but 186 00:10:23,840 --> 00:10:26,920 the agent is actually running the loop instead of the bash 187 00:10:26,920 --> 00:10:30,080 script. This is the primary difference 188 00:10:30,080 --> 00:10:32,240 right between the harness and the bash script. 189 00:10:33,960 --> 00:10:37,600 The problem is with the agent, sometimes you hit context limits 190 00:10:37,800 --> 00:10:40,800 right that it doesn't. It tries to do a little too 191 00:10:40,800 --> 00:10:44,400 much. You kind of notice that it maybe 192 00:10:44,400 --> 00:10:46,320 is going a little bit where you don't want to go and you 193 00:10:46,320 --> 00:10:49,880 intervene and then you blow out the contacts where is actually 194 00:10:49,880 --> 00:10:51,080 it would have been fine without you. 195 00:10:52,600 --> 00:10:54,640 But I honestly, I think this is going to be solved. 196 00:10:54,760 --> 00:11:00,240 Like give a, you know, give the agent labs three months and 197 00:11:00,240 --> 00:11:01,480 they're going to figure this out. 198 00:11:02,680 --> 00:11:04,840 You know, and I'm a big fan of AMP. 199 00:11:04,840 --> 00:11:08,720 Obviously I use the tool. I've decided to step back from 200 00:11:08,720 --> 00:11:11,440 the team and focus on my start up full time now. 201 00:11:11,440 --> 00:11:14,240 So I'm not on the amp team anymore, but I think it's a 202 00:11:14,240 --> 00:11:16,520 great product. I think, you know, the cloud 203 00:11:16,680 --> 00:11:18,840 code team is is thinking the same thing. 204 00:11:18,840 --> 00:11:23,680 I think cursor, I think Droid, I think everyone understands that 205 00:11:23,680 --> 00:11:27,920 we're moving to a new paradigm where we are all agent wranglers 206 00:11:28,560 --> 00:11:32,200 and engineering managers and there really isn't a place 207 00:11:32,200 --> 00:11:35,000 anymore for humans to write code like it. 208 00:11:35,520 --> 00:11:38,240 It really, it doesn't make sense. 209 00:11:38,720 --> 00:11:41,400 And this is coming from me, a guy who I got a computer science 210 00:11:41,400 --> 00:11:43,800 degree 25 years ago. I've taught a million people how 211 00:11:43,800 --> 00:11:47,600 to code at my last start up. Like it's weird. 212 00:11:47,840 --> 00:11:49,680 Yeah, we've crossed this threshold. 213 00:11:49,680 --> 00:11:52,200 I can't remember the last time I sat down and wrote an entire 214 00:11:52,200 --> 00:11:54,720 file myself, or more than just a line. 215 00:11:54,720 --> 00:11:57,000 I'd want to do a little tweak or edit, but that's about it. 216 00:11:58,000 --> 00:11:59,480 I know, I know. It's so weird. 217 00:11:59,480 --> 00:12:01,920 And, you know, sometimes you feel a little lazy, like you're 218 00:12:01,920 --> 00:12:05,160 asking, you know, the agent to do something you literally could 219 00:12:05,160 --> 00:12:08,440 do probably faster and quicker. But but then you kind of got to 220 00:12:08,440 --> 00:12:10,240 open up your ID and you got to open the file. 221 00:12:10,240 --> 00:12:11,600 And it's sort of like, forget it. 222 00:12:11,600 --> 00:12:14,880 Like I'll just spend, you know, 100 tokens to get it done. 223 00:12:15,280 --> 00:12:18,960 Last week we had Eric Busan and he did advanced Cloud code is 224 00:12:18,960 --> 00:12:21,080 the second time on a lot of great insight there. 225 00:12:21,240 --> 00:12:23,120 He's great. He was going, he's, he's 226 00:12:23,120 --> 00:12:26,400 awesome, but he went really hard on the hooks and convinced me. 227 00:12:26,400 --> 00:12:28,200 I've started implementing them more just to give me more 228 00:12:28,200 --> 00:12:29,760 confidence that's going to be done correctly. 229 00:12:29,960 --> 00:12:32,960 And I feel like a hook combined in a Ralph Wiggum loop will 230 00:12:32,960 --> 00:12:34,880 really be effective. But then you're starting to get 231 00:12:34,880 --> 00:12:37,720 more opinionated with it, trying to get more humans involved. 232 00:12:38,000 --> 00:12:41,240 Do you see room for these deterministic scripts being 233 00:12:41,280 --> 00:12:43,680 periodically called throughout these loops, or is that just 234 00:12:43,680 --> 00:12:47,560 kind of polluting the context? I think it's where we need to be 235 00:12:47,560 --> 00:12:51,920 right now. I don't think the models are 236 00:12:51,920 --> 00:12:58,120 quite good enough with, without the deterministic harnessing on 237 00:12:58,120 --> 00:12:59,880 top. And I actually think right right 238 00:12:59,880 --> 00:13:03,440 now we probably need more deterministic harnessing than 239 00:13:03,440 --> 00:13:05,920 people think to really get good outcomes. 240 00:13:05,960 --> 00:13:08,880 And, and, but I think it will fade. 241 00:13:09,000 --> 00:13:12,920 You know, it's my guess is that, you know, we'll probably see 242 00:13:12,920 --> 00:13:23,000 Opus 5, you know, Gemini 3.5, you know, GPT 5.5 or 6 and, and, 243 00:13:23,000 --> 00:13:26,240 and we'll realize we just don't need a lot of the, the 244 00:13:26,240 --> 00:13:31,320 deterministic scaffolding. However, I'm curious. 245 00:13:31,320 --> 00:13:34,000 Tell me more about the hooks and and what it's accomplishing. 246 00:13:34,320 --> 00:13:39,600 So the initial user submit query allows you to or. 247 00:13:39,840 --> 00:13:42,280 It's ideal for just making sure it has full context. 248 00:13:42,280 --> 00:13:44,960 So if this would be an ideal entry point for, make sure you 249 00:13:44,960 --> 00:13:47,520 have the To Do List. Make sure you have the scratch 250 00:13:47,520 --> 00:13:49,840 pad, just it reference there. And I feel like if it's all in 251 00:13:49,840 --> 00:13:51,040 the same directory, it's less important. 252 00:13:51,040 --> 00:13:53,720 But if you're kind of working from a a master copy somewhere 253 00:13:54,320 --> 00:13:57,400 and then later on the pre and post tool calls helps with 254 00:13:57,400 --> 00:13:59,160 reliability. Did the tool work successfully? 255 00:13:59,160 --> 00:14:00,480 Is the response something that's reasonable? 256 00:14:00,480 --> 00:14:03,520 Is it actually relevant? And those are more of the LM as 257 00:14:03,520 --> 00:14:06,280 a judge rather than just the the pure deterministic. 258 00:14:06,600 --> 00:14:08,080 And then at the end, the stop condition. 259 00:14:08,080 --> 00:14:10,960 This is where Eric really harped on making sure that you use 260 00:14:10,960 --> 00:14:14,280 adversarial reactions being like, if you were to look at 261 00:14:14,280 --> 00:14:17,200 this and try to poke every holy candidate, what would it do? 262 00:14:17,360 --> 00:14:19,720 Get the response there, feed it back in, see if it can correct 263 00:14:19,720 --> 00:14:22,400 or if it's just a dead end, and then start the loop over again. 264 00:14:22,800 --> 00:14:25,400 Interesting. I think that's that's great idea 265 00:14:25,400 --> 00:14:28,800 and he is such a pro. So I want to actually dig into 266 00:14:28,800 --> 00:14:33,400 that and look into it. There's a lot of of rounding of 267 00:14:33,400 --> 00:14:38,720 edges that we need to do right now where, you know, all of us 268 00:14:38,720 --> 00:14:42,440 that have been engineering for decades, you know, inherently 269 00:14:42,440 --> 00:14:46,600 know these things and and the models do not and they do need 270 00:14:46,600 --> 00:14:50,640 to be scaffolded in right now. But but what's interesting is 271 00:14:50,640 --> 00:14:54,240 like, it sounds like the basic idea between between these are 272 00:14:54,240 --> 00:14:58,800 behind these hooks and also the simple, you know, elements of 273 00:14:58,800 --> 00:15:04,240 the Ralph Wiggin loop, like read progress dot TST update it, you 274 00:15:04,240 --> 00:15:08,600 know, verify that the next user story is the right one. 275 00:15:08,960 --> 00:15:11,520 All these kind of things are sort of a loop within the loop. 276 00:15:11,560 --> 00:15:15,600 It's like, I would actually like to make a quick call, you know, 277 00:15:15,600 --> 00:15:18,240 to an LM as a judge to do more of that. 278 00:15:18,680 --> 00:15:21,600 So I think that that actually he's probably right on the money 279 00:15:21,600 --> 00:15:23,120 and you're right on the money in that. 280 00:15:23,360 --> 00:15:25,600 I'm just not sure how long we'll need to do it. 281 00:15:25,880 --> 00:15:31,440 And I, I feel like we all have this kind of anxiety or 282 00:15:31,440 --> 00:15:35,280 confusion or messiness about our workflow right now, right? 283 00:15:35,640 --> 00:15:41,400 We want to add in all of these deterministic programmatic ways 284 00:15:41,400 --> 00:15:44,000 to control the workflow. We know if we did, we would have 285 00:15:44,000 --> 00:15:47,480 better outcomes, but the ground is shifting underneath us so 286 00:15:47,480 --> 00:15:52,760 continually that you end up implementing maybe half of these 287 00:15:52,760 --> 00:15:56,040 ideas, right? Or or less because you think, Oh 288 00:15:56,040 --> 00:15:59,200 my God, like I'm just not going to really need to do this. 289 00:15:59,200 --> 00:16:02,400 I'm just kind of grind my way through it, you know, until the 290 00:16:02,400 --> 00:16:04,320 model just does it itself. I see I. 291 00:16:04,320 --> 00:16:07,360 Mean, I remember I first heard of the Ralph Wiggum loop from 292 00:16:07,360 --> 00:16:10,200 Dexter Horthy, I don't know, six months ago, like I felt like it 293 00:16:10,200 --> 00:16:12,240 was ages, but I was like, there's too many changes. 294 00:16:12,240 --> 00:16:14,840 I am not going to implant it till there's a little more, you 295 00:16:14,840 --> 00:16:17,560 know, force behind it. And it's just, it kept 296 00:16:17,560 --> 00:16:19,880 snowballing to something big. Your blog post went pretty 297 00:16:19,880 --> 00:16:22,440 viral. So it's just it has been battle 298 00:16:22,440 --> 00:16:24,920 tested and proven that this is a valid strategy for this point in 299 00:16:24,920 --> 00:16:27,200 time with the current state of capability. 300 00:16:27,200 --> 00:16:32,120 For, for this like one month in time and a little, you know, 301 00:16:32,120 --> 00:16:37,200 behind the scenes on that post. So I started to see people talk 302 00:16:37,200 --> 00:16:39,640 about the Roth Wiggum loop. I thought, you know what, I 303 00:16:39,680 --> 00:16:42,720 should really give it a go like I'm curious how good it is. 304 00:16:43,280 --> 00:16:46,160 I'm so of course I fired up amp and I was like, OK, you know, 305 00:16:46,160 --> 00:16:48,800 here's Jeff's post like help me set this up. 306 00:16:49,120 --> 00:16:52,160 And of course it, you know, I set up the bash script. 307 00:16:52,160 --> 00:16:54,120 It figured out the system prompt. 308 00:16:54,120 --> 00:16:57,280 We iterate a little bit man and, and kicked it off and then 309 00:16:57,280 --> 00:17:00,720 after, and then I, I think I shipped 2 features with it and I 310 00:17:00,720 --> 00:17:04,040 was like, cool, OK. And then something beside was 311 00:17:04,040 --> 00:17:05,359 like, you should write about that. 312 00:17:05,839 --> 00:17:08,720 And so I just went back in the same thread in the amp and I was 313 00:17:08,720 --> 00:17:12,280 like, Hey amp, why do you write A blog post about what we just 314 00:17:12,280 --> 00:17:15,880 did and how we did it? And so the other people could 315 00:17:15,880 --> 00:17:18,359 set it up too and make it a markdown because I'm going to 316 00:17:18,359 --> 00:17:21,240 post it on an X because I, I could tell articles are starting 317 00:17:21,240 --> 00:17:25,720 to get juiced by the algo an X. And so it pumped out, you know, 318 00:17:25,720 --> 00:17:29,080 pretty good markdown. I copied and pasted it into X 319 00:17:29,080 --> 00:17:32,800 and then I edit it and massage it a bit and change the, the 320 00:17:32,800 --> 00:17:34,520 title to be a little bit more hooky. 321 00:17:34,880 --> 00:17:37,520 And then I hit post and it, you know, you never know. 322 00:17:39,000 --> 00:17:41,960 And then and then, but then you know, as soon as it hits, right? 323 00:17:41,960 --> 00:17:45,080 Because I was like holy biscuits, like all of a sudden 324 00:17:45,080 --> 00:17:49,160 it had 50,000 views and then it was like 200,000 views. 325 00:17:49,160 --> 00:17:51,400 And I was like, oh, I think this is going to the moon. 326 00:17:52,080 --> 00:17:55,160 And it's, it's like 1.8 million views now on that little 327 00:17:55,160 --> 00:17:56,600 article. It's hilarious. 328 00:17:57,160 --> 00:17:59,960 So good one area. That I'd like to explore a 329 00:17:59,960 --> 00:18:01,640 little bit more is when you mentioned the acceptance 330 00:18:01,640 --> 00:18:05,840 criteria, it feels like one of those things where if if you're 331 00:18:05,840 --> 00:18:08,040 off by, it's like going to the moon, off by an inch, you're 332 00:18:08,040 --> 00:18:10,440 going to miss it completely. You need to be pretty precise. 333 00:18:10,480 --> 00:18:13,560 And if you just let Ralph go overnight, I feel like bad 334 00:18:13,560 --> 00:18:15,320 acceptance criteria could lead you down the wrong path. 335 00:18:15,320 --> 00:18:16,600 Have you experienced that to be true? 336 00:18:16,600 --> 00:18:19,520 Is that like an unfounded very much? 337 00:18:19,680 --> 00:18:24,200 Oh my God, very much so. I, so mostly I'm building a web 338 00:18:24,200 --> 00:18:26,280 app, right? So my startup is called 339 00:18:26,280 --> 00:18:28,760 Untangle. It is a divorce AI agent. 340 00:18:28,760 --> 00:18:32,720 It's a next JS app. And you know, it's a lot, it's, 341 00:18:32,920 --> 00:18:39,120 it's mostly web UI, right? And so I need a lot of good web 342 00:18:39,520 --> 00:18:44,160 interface testing, right? And so I, I was really 343 00:18:44,160 --> 00:18:47,640 struggling, you know, it's sort of hodgepodging, you know, 344 00:18:49,640 --> 00:18:55,360 Chrome Dev tools, MCP with some custom, you know, skills with 345 00:18:55,360 --> 00:18:58,320 all sorts of stuff. And, and I would have acceptance 346 00:18:58,320 --> 00:19:01,200 criteria around browser testing. And this is like clearly 347 00:19:01,200 --> 00:19:04,440 important. But as we all know, like browser 348 00:19:04,440 --> 00:19:07,880 testing and front end UI stuff is just really hard with agents. 349 00:19:08,480 --> 00:19:11,440 But I think as soon as Versailles shipped agent 350 00:19:11,440 --> 00:19:15,040 browser, it has been a big unlock. 351 00:19:16,200 --> 00:19:20,560 So it's basically you, you install it and then there's a 352 00:19:20,560 --> 00:19:25,200 skill attached to it and it has really unlocked browser testing 353 00:19:25,200 --> 00:19:26,920 for me as an acceptance criteria. 354 00:19:27,560 --> 00:19:28,960 And I'm not being paid to say that. 355 00:19:29,000 --> 00:19:30,560 I'm not being paid to say anything here. 356 00:19:30,760 --> 00:19:34,000 And it's very good. So I would, I would, I would 357 00:19:34,000 --> 00:19:38,560 recommend people dig into that. And then you specify that when 358 00:19:38,560 --> 00:19:41,160 you have acceptance criteria that involve front end testing 359 00:19:41,560 --> 00:19:45,360 that it should use, you know, XYZ, whatever it is. 360 00:19:45,360 --> 00:19:47,520 I say you have to use agent browser. 361 00:19:49,160 --> 00:19:53,000 And then it's much better, you know, and then I realized, wait 362 00:19:53,000 --> 00:19:56,520 a minute, like 10 of these user stories at the end of this loop, 363 00:19:56,840 --> 00:20:02,160 you know, should be browser testing or very edge KC, you 364 00:20:02,160 --> 00:20:06,360 know, user test because you can't like, I think often we go 365 00:20:06,360 --> 00:20:09,160 back to sort of a human instincts, which is, Oh my God, 366 00:20:09,320 --> 00:20:11,160 testing is so expensive and hard. 367 00:20:11,520 --> 00:20:14,640 I'm going to do the happy path plus like a couple edge cases, 368 00:20:15,160 --> 00:20:16,920 but you don't need to do that anymore. 369 00:20:16,920 --> 00:20:20,240 You can do the happy path plus 100 edge cases like if you're 370 00:20:20,240 --> 00:20:24,800 happy to pay for the tokens. So in my Ralph William loop, I 371 00:20:24,800 --> 00:20:30,720 actually have a skill to create the PRD and the user stories for 372 00:20:30,720 --> 00:20:33,720 Ralph loop. And I've actually wound that 373 00:20:33,720 --> 00:20:38,040 into the skill itself. So I just fire, I invoke the 374 00:20:38,040 --> 00:20:42,760 skill, and then the skill says, OK, are you wanting to create a 375 00:20:42,760 --> 00:20:46,440 new PRD for a Ralph Loop or you did you already create 1? 376 00:20:46,440 --> 00:20:47,840 You want me to execute the tasks? 377 00:20:48,240 --> 00:20:51,760 So I'll say #1 you know, and then it, it walks me through, 378 00:20:52,040 --> 00:20:55,120 you know, pretty standard prompt around creating a PRD. 379 00:20:55,560 --> 00:20:59,360 And then after that, it starts fleshing out the user stories. 380 00:20:59,400 --> 00:21:03,000 And in those in that prompt to build up these user stories, 381 00:21:03,000 --> 00:21:06,480 it's like you have to refine and refine and refine those user 382 00:21:06,480 --> 00:21:08,120 stories with good acceptance criteria. 383 00:21:08,240 --> 00:21:10,400 And this is where you just can't fall asleep at the switch. 384 00:21:10,400 --> 00:21:15,320 Like I think defining a good Ralph Wiggum loop takes an hour 385 00:21:15,320 --> 00:21:17,400 minimum. You know, this is not a 20 386 00:21:17,400 --> 00:21:19,240 minute thing. You know, you need to grab a 387 00:21:19,240 --> 00:21:25,720 coffee and engage your brain and really write a good PRD and good 388 00:21:25,720 --> 00:21:28,760 user stories and then and then look at them and see if they're 389 00:21:28,800 --> 00:21:32,440 actually testable by an agent without any in the loop. 390 00:21:33,360 --> 00:21:37,320 Otherwise you just end up at the end having to spend half a day, 391 00:21:37,880 --> 00:21:42,000 you know, fixing things that that were poorly specified and 392 00:21:42,000 --> 00:21:43,440 that really. Does feel like the latest 393 00:21:43,440 --> 00:21:46,640 developer skill where you have to be able to think through as 394 00:21:46,680 --> 00:21:49,880 as concretely and thoroughly as possible to create these 395 00:21:49,880 --> 00:21:52,640 documents, create these even just conversations with these AI 396 00:21:52,640 --> 00:21:54,120 agents in order to get the results you want. 397 00:21:54,360 --> 00:21:56,800 Yeah, and. This is where, you know, I think 398 00:21:56,800 --> 00:22:01,160 we're all still a little baffled at the lack of common sense with 399 00:22:01,160 --> 00:22:02,960 the models. You know it. 400 00:22:03,000 --> 00:22:06,680 It's one of those things where you still, I'm still surprised 401 00:22:06,680 --> 00:22:11,040 sometimes where I forget to say something, you know, to specify 402 00:22:11,040 --> 00:22:15,360 something because it just seems so damn obvious to me, you know, 403 00:22:15,960 --> 00:22:19,600 that you forget and, and we're still not even even though these 404 00:22:19,600 --> 00:22:22,840 things are in my agent dot MD files, like I've done the work 405 00:22:22,840 --> 00:22:25,240 people like I'm not shortcutting here. 406 00:22:25,800 --> 00:22:28,520 There's just a couple times where you're like, how did you 407 00:22:28,520 --> 00:22:32,840 not know that? You know, so we're still there. 408 00:22:32,840 --> 00:22:35,240 And it feels like you should be able to solve that with another 409 00:22:35,240 --> 00:22:36,800 loop. Like surely I could have another 410 00:22:36,800 --> 00:22:40,560 loop which looks at my loop, which checks my loop, you know, 411 00:22:40,920 --> 00:22:45,000 which checks its loop. But it sometimes it just, you 412 00:22:45,000 --> 00:22:46,880 just need a human still. Yep, Yep. 413 00:22:47,040 --> 00:22:48,600 Can't quite be loops all the way down yet. 414 00:22:49,560 --> 00:22:51,560 You mentioned agent browser, which is really cool. 415 00:22:51,640 --> 00:22:54,840 Are there any other tools or libraries you recommend or 416 00:22:54,880 --> 00:22:57,520 experimented with that help make the Ralph Wiggle loop more 417 00:22:57,520 --> 00:22:59,280 reliable? Not specifically. 418 00:22:59,280 --> 00:23:05,600 Ralph I would say I'm using anthropics front end designer 419 00:23:05,600 --> 00:23:09,040 skill a lot. It's just a great skill. 420 00:23:09,200 --> 00:23:13,800 Like it really often takes kind of a sloppy, you know, AIH and 421 00:23:13,800 --> 00:23:18,760 TUI and really cleans it up in a really nice way. 422 00:23:19,000 --> 00:23:21,520 And sometimes you just need a nice deck or you need a nice, 423 00:23:21,600 --> 00:23:24,400 you know, holding landing page for something. 424 00:23:24,600 --> 00:23:28,600 Just don't want it to look AIE. So that one's good. 425 00:23:29,520 --> 00:23:32,560 Also, I, I keep singing the, the praises for sale. 426 00:23:32,560 --> 00:23:36,520 I'm, I'm not paid by anybody to say any things, but they just 427 00:23:36,520 --> 00:23:42,880 released a specific set of skills for React for next JS and 428 00:23:43,120 --> 00:23:47,640 you, you know, MPM install them and have all these magic skills. 429 00:23:47,640 --> 00:23:50,000 So I, I think skills are really where it's at right now. 430 00:23:51,000 --> 00:23:53,000 And because they're progressively discovered as 431 00:23:53,000 --> 00:23:54,800 well, they're not polluting your context. 432 00:23:54,800 --> 00:23:57,320 And you know, I'm pretty bullish on those. 433 00:23:57,320 --> 00:24:01,000 So those are a couple of the, the things I'm using more of 434 00:24:01,240 --> 00:24:03,400 that. I have actually a lot of FOMO 435 00:24:03,400 --> 00:24:07,560 right now because I don't have a Mac mini sitting here running 436 00:24:07,560 --> 00:24:12,000 some sort of clawed loop. And I've I've just heard 437 00:24:12,000 --> 00:24:17,000 everybody raving about clawed, clawed bot. 438 00:24:17,000 --> 00:24:20,640 What what's Pete's thing called? I'm not sure. 439 00:24:21,080 --> 00:24:23,920 Claw dot. This is the problem. 440 00:24:23,920 --> 00:24:24,960 These are these things I need to know. 441 00:24:25,280 --> 00:24:29,600 There's this. Basically the idea is that Pete 442 00:24:29,600 --> 00:24:33,040 Steinberg has released this framework where you basically 443 00:24:33,040 --> 00:24:37,600 fire up a Mac mini or, or, and then you install cloud on it and 444 00:24:37,600 --> 00:24:41,200 it can basically do anything and it becomes this assistant for 445 00:24:41,200 --> 00:24:42,680 you. And I'm like, I'm just not doing 446 00:24:42,680 --> 00:24:43,840 enough. I need to do more. 447 00:24:44,600 --> 00:24:46,360 Yeah. That actually reminds me, back 448 00:24:46,360 --> 00:24:48,360 in the Open Interpreter days, someone took a Mac mini, 449 00:24:48,360 --> 00:24:51,560 installed Open Interpreter on it, gave itself its own e-mail 450 00:24:51,560 --> 00:24:54,200 address, its own iMessage, and it could just communicate 451 00:24:54,200 --> 00:24:56,200 through your phone to this little device, have it do 452 00:24:56,200 --> 00:24:57,640 anything you want. So yeah, we're going to see 453 00:24:57,640 --> 00:25:01,480 these magical things. I actually just a little Ricci 454 00:25:01,480 --> 00:25:04,960 mini setup. Cool, I want one of those so 455 00:25:04,960 --> 00:25:06,360 there will be more videos on that coming. 456 00:25:06,640 --> 00:25:08,840 This should be over your. Shoulder like tearing aside. 457 00:25:08,880 --> 00:25:09,960 Yeah, I want to get in the background. 458 00:25:09,960 --> 00:25:12,640 But also, it's going to be my little like accountability 459 00:25:12,640 --> 00:25:14,880 buddy, just making sure I'm like getting up from my desk enough. 460 00:25:14,880 --> 00:25:17,680 It helped me play in my day. So I have a lot of ambition 461 00:25:17,680 --> 00:25:19,720 work. But yeah, just hardware just 462 00:25:19,720 --> 00:25:20,960 offsetting. And The thing is, sorry, the 463 00:25:20,960 --> 00:25:23,360 reason I brought that up is it has a Raspberry pie on board, 464 00:25:23,360 --> 00:25:24,960 very limited in terms of processing and power. 465 00:25:24,960 --> 00:25:28,560 Can still do some cool stuff but I have a MacBook with 96 gigs of 466 00:25:28,640 --> 00:25:31,160 unified ram so I just bring a little live kit tunnel. 467 00:25:31,160 --> 00:25:33,880 So now I can have all the heavy processing my Mac, keep it fully 468 00:25:33,880 --> 00:25:36,800 local but I still get the camera into bi direction communications 469 00:25:36,800 --> 00:25:38,280 from the robot. What a time to. 470 00:25:38,280 --> 00:25:41,560 Be alive and and like I know you and I would never do these 471 00:25:41,560 --> 00:25:44,000 things because it would just be too much work to code it without 472 00:25:44,000 --> 00:25:46,120 agents. But now, you know, you could 473 00:25:46,120 --> 00:25:48,520 probably crank that out in like an hour and that that would be 474 00:25:48,520 --> 00:25:50,840 like a whole year project before. 475 00:25:51,160 --> 00:25:52,560 Yeah, because. It's not just the actual 476 00:25:52,560 --> 00:25:54,280 building, it's learning everything you need, all the 477 00:25:54,280 --> 00:25:56,680 prerequisites coming up to it. Yeah, I. 478 00:25:57,120 --> 00:25:59,720 You know, I had a dinner recently and SF with some 479 00:25:59,720 --> 00:26:03,600 friends and, and we were all just marveling how much fun 480 00:26:03,600 --> 00:26:08,880 we're having, you know, and it feels like, it feels opposite of 481 00:26:08,880 --> 00:26:13,520 the impending doom that we all know is sort of coming for 482 00:26:13,520 --> 00:26:16,480 software engineering and all of the media hype that we're 483 00:26:16,480 --> 00:26:19,200 hearing about the traditional sweet roll. 484 00:26:19,320 --> 00:26:22,040 And it, it just makes me think that we're missing something 485 00:26:22,160 --> 00:26:27,400 that we're missing, you know, the, just the explosion of 486 00:26:27,400 --> 00:26:31,760 creation that's going to happen and the joy that comes from 487 00:26:31,760 --> 00:26:34,040 that. I was saying, you know, I'm 488 00:26:34,040 --> 00:26:37,840 micro gaming now. Like I, I'm 48, I've got kids 489 00:26:37,840 --> 00:26:40,360 that are old. I, I literally haven't games for 490 00:26:40,360 --> 00:26:44,840 like 18 years because I don't have time like between being 491 00:26:44,840 --> 00:26:49,360 ACEO and a dad and a husband, you know, I've got no time. 492 00:26:49,840 --> 00:26:53,160 And the other day obviously, you know, I'm waiting for amp to do 493 00:26:53,160 --> 00:26:56,920 something that's like 2 minutes and I can check X or I can play 494 00:26:56,920 --> 00:26:59,920 a game. And so Andrew Mileage who works 495 00:26:59,920 --> 00:27:03,040 at cursor, he created this open source game called ISO City, 496 00:27:03,280 --> 00:27:07,680 which is basically SIM City. And you know, I fired up in a 497 00:27:07,680 --> 00:27:09,960 browser, I play it. I committed some code to it 498 00:27:09,960 --> 00:27:13,200 which got merged. And so, you know, I'm doing that 499 00:27:13,200 --> 00:27:15,720 more now, and that's only because of agents like. 500 00:27:16,400 --> 00:27:18,720 So I'm very I'm very excited about the future. 501 00:27:18,720 --> 00:27:20,640 I think it's going to be a lot. I think it's a lot of fun. 502 00:27:21,120 --> 00:27:21,720 Yeah. I. 503 00:27:21,720 --> 00:27:24,720 Think there was a tweet put out recently by Kyle Corbett saying 504 00:27:24,720 --> 00:27:28,400 he really struggles to see a future where the 50 percentile 505 00:27:28,840 --> 00:27:31,640 AI or a software engineer has has a role. 506 00:27:32,000 --> 00:27:34,480 But I think that's just viewing it in the lens of what software 507 00:27:34,480 --> 00:27:36,000 engineering has been up to this point. 508 00:27:36,160 --> 00:27:38,880 But when you get into the the idea of orchestrating and 509 00:27:38,880 --> 00:27:40,520 problem solving, none of that changes. 510 00:27:40,520 --> 00:27:42,720 It's just we're no longer writing syntax and we're able to 511 00:27:42,760 --> 00:27:46,840 amplify our delivery speed so much that we're just creating, 512 00:27:47,080 --> 00:27:48,720 creating value in, in one way or another. 513 00:27:48,880 --> 00:27:51,280 Yeah, it feels. Like we're magicians now, you 514 00:27:51,280 --> 00:27:56,240 know, you, you really, you have a wand now, whereas before you 515 00:27:56,240 --> 00:28:00,200 had sort of a hammer, you know, and some nails and, and it was, 516 00:28:00,240 --> 00:28:03,600 it was harder and slower, whereas now you just wave your 517 00:28:03,600 --> 00:28:07,960 wand and, and it still takes creativity and, and hard work, 518 00:28:07,960 --> 00:28:10,320 but it's just, it's so much more fun. 519 00:28:11,440 --> 00:28:14,560 So one other thing I want to mention that I've learned and, 520 00:28:14,560 --> 00:28:16,640 and I think a lot of people are getting this as well in the, in 521 00:28:16,640 --> 00:28:19,680 the vein of loops, is that there's a lot of other areas you 522 00:28:19,680 --> 00:28:27,200 can apply a loop to your life. And so we have, so for my 523 00:28:27,200 --> 00:28:29,960 business, we have lead capture tools, right? 524 00:28:29,960 --> 00:28:31,640 So when you're thinking about divorce, you're probably 525 00:28:31,640 --> 00:28:33,480 thinking about child support, right? 526 00:28:33,560 --> 00:28:35,360 You know, like how much child support am I going to get or 527 00:28:35,360 --> 00:28:38,160 pay, right? So we have a, a form people fill 528 00:28:38,160 --> 00:28:42,600 out and then of the details and then at the end they give us 529 00:28:42,600 --> 00:28:44,920 their e-mail and then obviously they go into an e-mail drip, 530 00:28:45,040 --> 00:28:47,920 right? A, a fun kind of extra thing 531 00:28:47,920 --> 00:28:50,080 that's happening there. And I, I just talked about this 532 00:28:50,960 --> 00:28:53,960 on, on Andrew's show was how those emails get written. 533 00:28:53,960 --> 00:28:55,360 But I'll, I'll leave that for his show. 534 00:28:55,480 --> 00:28:58,240 But now we've got click data, right? 535 00:28:58,240 --> 00:29:01,320 So we, we know if people are clicking on these emails because 536 00:29:01,320 --> 00:29:03,720 it's in my database, right? And that's a whole other thing 537 00:29:03,720 --> 00:29:08,120 is like, basically I've built an entire SAS tool, you know, for 538 00:29:08,120 --> 00:29:13,120 my e-mail marketing pipeline. But hey, and so obviously all 539 00:29:13,120 --> 00:29:15,520 the click data is going to my database, which I love because I 540 00:29:15,520 --> 00:29:17,680 own it. And I was thinking about it and 541 00:29:17,680 --> 00:29:22,320 the other day I'm like, I really would love to, to know kind of 542 00:29:22,320 --> 00:29:23,760 what's happening here with this data. 543 00:29:23,760 --> 00:29:26,320 And I was like, well, duh, I should just ask, you know, Opus 544 00:29:26,320 --> 00:29:29,680 4-5. And So what I did is I set up a 545 00:29:29,680 --> 00:29:35,520 Cron job where basically I, I pull in the right contacts from 546 00:29:35,520 --> 00:29:37,040 the database. You know, here's the e-mail 547 00:29:37,040 --> 00:29:40,480 drips, here's the number of clicks, here's the conversion 548 00:29:40,480 --> 00:29:44,440 rate from each step. Take all that data and give me 549 00:29:44,440 --> 00:29:47,760 some insight and then e-mail it to me in the morning, right? 550 00:29:48,160 --> 00:29:51,720 So every morning I wake up with insight, it's like, hey, here's 551 00:29:51,720 --> 00:29:54,080 what happened. You know, with the e-mail drips, 552 00:29:54,080 --> 00:29:55,720 here's what what was clicked on here? 553 00:29:55,720 --> 00:29:57,720 What was it? Here's something you should do 554 00:29:57,720 --> 00:29:59,960 about it. And it's like, that's something 555 00:29:59,960 --> 00:30:03,000 I would used to have to pay AVP of marketing $200,000 a year 556 00:30:03,000 --> 00:30:05,760 for. And I just got that for $0.15. 557 00:30:08,000 --> 00:30:10,480 So that's fun. But then it loops again, right? 558 00:30:10,480 --> 00:30:16,560 So then one of the suggestions from Opus 4-5 was like, well, 559 00:30:16,720 --> 00:30:20,320 you know, these four people have clicked on more than 3 emails. 560 00:30:20,360 --> 00:30:22,760 You should probably do something about that, you know, like, why 561 00:30:22,760 --> 00:30:26,080 don't you reach out to them? And my God, that's smart Opus. 562 00:30:26,280 --> 00:30:28,480 Thank you. And I was like, oh, well, let's 563 00:30:28,480 --> 00:30:31,320 create another loop, right? So I was like, great, now I want 564 00:30:31,320 --> 00:30:35,400 you to create another Cron job that based on that report, if it 565 00:30:35,400 --> 00:30:38,320 finds anyone's e-mail, anyone's emails that they've clicked on 566 00:30:38,320 --> 00:30:40,560 something more than three times, I want you to put them on 567 00:30:40,560 --> 00:30:43,400 another e-mail drip. That's an e-mail from me 568 00:30:43,400 --> 00:30:46,920 inviting them to a phone call, you know, wrote it, committed 569 00:30:46,920 --> 00:30:48,920 it, pushed it. And the next morning I woke up 570 00:30:48,920 --> 00:30:52,640 to an e-mail reply from one of our customers on the e-mail drip 571 00:30:52,640 --> 00:30:57,520 asking to meet me, right. So the, the everyone should be 572 00:30:57,520 --> 00:31:03,240 thinking about the loops, right? And how you can compound the the 573 00:31:03,240 --> 00:31:07,680 knowledge into wisdom into action, because it's basically 574 00:31:07,680 --> 00:31:11,240 free, right? You know, it's $0.15 a loop now. 575 00:31:12,120 --> 00:31:15,720 So how can you use that to actually create action? 576 00:31:15,960 --> 00:31:18,840 Because the other problem is you, you end up with, you know, 577 00:31:18,960 --> 00:31:21,320 analysis process like just too much information. 578 00:31:21,560 --> 00:31:25,560 And so that's why I have another loop run, synthesize information 579 00:31:25,560 --> 00:31:29,360 to an action, and then maybe even trigger a Ralph loop to 580 00:31:29,360 --> 00:31:31,840 implement the action and then wake up to APR instead. 581 00:31:32,280 --> 00:31:33,040 Very. Cool. 582 00:31:33,200 --> 00:31:39,240 Do you have any guidance or experience on when a task is 583 00:31:39,240 --> 00:31:41,400 either too simple for a loop or how? 584 00:31:41,400 --> 00:31:43,960 Because kind of I think of the GPT 5 approach, you can send it 585 00:31:43,960 --> 00:31:45,440 any request and it chooses the model. 586 00:31:45,440 --> 00:31:49,200 Is there an A heuristic or a plan that you have where for a 587 00:31:49,200 --> 00:31:53,640 given task, you know, loop it single one or like super hard 588 00:31:53,640 --> 00:31:55,040 thing type thing? I wish I. 589 00:31:55,040 --> 00:31:56,960 Did I wish it was more scientific? 590 00:31:57,160 --> 00:32:01,080 It's totally a gut feeling, you know, and again, this is kind of 591 00:32:01,080 --> 00:32:03,320 where when you have experience, you know, shipping much stuff 592 00:32:03,320 --> 00:32:05,960 for a bunch of years, you kind of concise things in your head 593 00:32:05,960 --> 00:32:07,640 like, oh, this is small, medium or large. 594 00:32:08,960 --> 00:32:13,200 And so no, I don't have any fancy, fancy method. 595 00:32:13,200 --> 00:32:16,440 Probably you should put it, you know, include that in your Ralph 596 00:32:16,440 --> 00:32:20,960 skill, where as you're chatting through a PRD, you know, it 597 00:32:20,960 --> 00:32:22,680 determines that for you. But then the problem is you have 598 00:32:22,680 --> 00:32:25,200 to decide to start the Ralph loop, which means you're 599 00:32:25,200 --> 00:32:26,680 probably tackling something large. 600 00:32:26,920 --> 00:32:29,720 It might be a good idea to build a skill that actually is called 601 00:32:29,720 --> 00:32:33,280 like, you know, sizing, which it sort of pipes up. 602 00:32:33,280 --> 00:32:35,640 And it's kind of like this is this is too much. 603 00:32:36,040 --> 00:32:38,480 This is too much to bite off. Like we should be breaking this 604 00:32:38,480 --> 00:32:41,680 up into a loop. Do you take inspiration from 605 00:32:41,680 --> 00:32:44,080 certain skills out in the wild? Do you just like get the idea? 606 00:32:44,080 --> 00:32:45,560 I would like it to do this better. 607 00:32:45,560 --> 00:32:47,240 So write a skill for it the only. 608 00:32:47,320 --> 00:32:49,680 Skill I've written was the Ralph skill so far. 609 00:32:49,680 --> 00:32:53,760 The rest of them I just steal so well, I actually bought some the 610 00:32:53,760 --> 00:32:56,600 other day. So the boring marketer who's a 611 00:32:56,600 --> 00:33:00,600 great AI marketer, he actually had a, a, a list of skills that 612 00:33:00,600 --> 00:33:03,720 I paid 200 bucks for and, and they were absolutely worth it. 613 00:33:03,720 --> 00:33:07,160 One was sort of how to write compelling e-mail drips. 1 was 614 00:33:07,160 --> 00:33:11,320 like market copy that converts. So I just bought those. 615 00:33:12,040 --> 00:33:15,080 But I would say most of it is beyond X. 616 00:33:15,160 --> 00:33:19,600 And then seeing smart people that I I respect say I use this 617 00:33:19,600 --> 00:33:26,080 skill and then I usually try it. Hence being the agent browser 618 00:33:27,480 --> 00:33:30,600 was, you know, I was following a bunch of smart people and I said 619 00:33:30,600 --> 00:33:35,240 that was good. So I mainly just copy and steal. 620 00:33:35,600 --> 00:33:38,000 What what? But not steal illegally. 621 00:33:38,120 --> 00:33:42,200 Nice steal. Yeah, and actually the the idea 622 00:33:42,200 --> 00:33:44,840 of paying for skills does make sense because you know you're 623 00:33:44,840 --> 00:33:47,200 going to get value out of it. When you took a look at these 624 00:33:47,200 --> 00:33:50,160 skills, was there anything, without going to specifics, was 625 00:33:50,160 --> 00:33:52,880 there anything that surprised you or was it just really like a 626 00:33:52,880 --> 00:33:56,280 distillation of his experience and expertise kind of put into a 627 00:33:56,280 --> 00:33:57,280 skill? It didn't. 628 00:33:57,280 --> 00:33:58,720 Surprise me. It was just a, a good 629 00:33:58,720 --> 00:34:01,360 distillation. And I think that's probably 630 00:34:01,360 --> 00:34:05,440 mostly what's going on here is that these are all things that 631 00:34:05,480 --> 00:34:09,080 either we know vaguely or we don't remember that we, we 632 00:34:09,080 --> 00:34:12,840 theoretically could work with, you know, Claude or AMP or or 633 00:34:13,520 --> 00:34:17,960 GPT 52 to, to write our skills. But the problem is you, 634 00:34:17,960 --> 00:34:21,760 sometimes you don't know to even ask to make the skill right. 635 00:34:21,920 --> 00:34:25,440 So it, it's kind of interesting as one of my previous businesses 636 00:34:25,440 --> 00:34:30,920 before Treehouse was an, an in person training company. 637 00:34:30,920 --> 00:34:33,280 So we would actually do workshops in person. 638 00:34:33,320 --> 00:34:38,679 You know how to build a PHP enterprise web app and people 639 00:34:38,679 --> 00:34:41,199 would pay to go to these one day conferences to learn a chunk of 640 00:34:41,199 --> 00:34:45,920 information and it's it's not something they couldn't figure 641 00:34:45,920 --> 00:34:49,760 out by just reading a an O'Reilly book, right. 642 00:34:49,760 --> 00:34:53,600 But the teacher, you know, distilling it when I think the 643 00:34:53,600 --> 00:34:57,080 word you use is good, like they distill it down to a chunk of 644 00:34:57,080 --> 00:35:01,040 information can excuse me, you can understand and absorb and 645 00:35:01,040 --> 00:35:03,480 then action. And I think that's what what's 646 00:35:03,480 --> 00:35:05,640 happening with skills or packs of skills. 647 00:35:06,760 --> 00:35:11,840 The the problem is I and I almost mistakenly open sourced 648 00:35:11,880 --> 00:35:14,040 the boring marketer skills. I felt terrible. 649 00:35:14,280 --> 00:35:17,520 I like put them in my skill directory, which is a repo. 650 00:35:18,040 --> 00:35:21,360 And I open sourced that because I wanted people to see my amp 651 00:35:21,360 --> 00:35:22,720 skills. And I was like, oh shit. 652 00:35:22,720 --> 00:35:26,160 Like his skills aren't. So I had to, so I, I had to 653 00:35:26,160 --> 00:35:28,200 remove them. So the problem is they're almost 654 00:35:28,200 --> 00:35:32,080 too shareable. I don't know how well the IP is 655 00:35:32,080 --> 00:35:34,600 protected because they're literally just text files. 656 00:35:35,560 --> 00:35:38,720 Yeah. But I think curation is often 657 00:35:38,720 --> 00:35:40,680 the value. I mean, why is your show great? 658 00:35:41,160 --> 00:35:43,600 You know, because you curate the content, right? 659 00:35:43,640 --> 00:35:48,040 So I think people are going to start to pay for that that taste 660 00:35:48,040 --> 00:35:52,600 that curation more as we see a proliferation of knowledge. 661 00:35:53,040 --> 00:35:56,200 I agree. Do you still use MCP at all or 662 00:35:56,200 --> 00:35:58,800 his skills kind of replace that I do? 663 00:35:58,800 --> 00:35:59,560 Adding. Information. 664 00:36:00,080 --> 00:36:02,440 So I used. To MCPS a lot. 1 is century. 665 00:36:02,440 --> 00:36:05,280 I love it. So Sentry has a great MCP where 666 00:36:05,280 --> 00:36:08,320 you're like, hey, look at this issue or what issues do we have? 667 00:36:08,880 --> 00:36:11,280 And I just, it's just really good and Sentry is amazing. 668 00:36:11,280 --> 00:36:14,920 And again, not being paid to say any of this, I just love Sentry. 669 00:36:14,960 --> 00:36:20,480 And I use the Fizzy MCP. So 37 signals has a simple 670 00:36:20,480 --> 00:36:25,120 Kanban board called Fizzy. And I was using like linear and 671 00:36:25,120 --> 00:36:26,720 I was like, Oh my God, it's too much. 672 00:36:26,840 --> 00:36:28,960 Like I do not need this much functionality. 673 00:36:30,560 --> 00:36:32,880 And then I was like, oh, I'll just use like markdown, like, 674 00:36:32,880 --> 00:36:35,400 why do I even need, you know, a Kanban board? 675 00:36:35,480 --> 00:36:39,840 But I, I kind of like a nice UI sometimes, you know, and so 676 00:36:39,840 --> 00:36:43,440 fizzy dot do is, is their free Kanban board tool and it's got 677 00:36:43,440 --> 00:36:45,520 an MCP. And so I just use it to say, 678 00:36:45,520 --> 00:36:48,680 hey, you know, is there any tasks right now that that you 679 00:36:48,680 --> 00:36:51,360 can knock out? That's the only two MCPS I I 680 00:36:51,360 --> 00:36:54,560 use. I did use Chrome Dev tools, but 681 00:36:54,560 --> 00:36:57,280 now that I have agent browser I don't need it specifically 682 00:36:58,000 --> 00:37:02,000 because it's wound in, you know, via playwright and that skill. 683 00:37:02,000 --> 00:37:04,680 So those are the only two MCPS I use. 684 00:37:05,240 --> 00:37:06,400 How about you? Very few. 685 00:37:06,400 --> 00:37:09,400 Actually I still use linear. I think part of it is to make 686 00:37:09,400 --> 00:37:11,360 sure the team stays engaged with linear. 687 00:37:11,360 --> 00:37:14,160 So Levi example I'm using it make sure you can't too. 688 00:37:15,200 --> 00:37:19,000 I actually, so I was leveraging the sentry API because I have 689 00:37:19,000 --> 00:37:21,920 this theory that you can build a pretty robust self healing 690 00:37:21,920 --> 00:37:25,720 system by taking the trace log or or the stack trace from 691 00:37:25,720 --> 00:37:27,760 sentry feeding it in and trying to have APR generated. 692 00:37:27,920 --> 00:37:30,000 Haven't totally got that running successfully yet. 693 00:37:30,000 --> 00:37:32,480 The idea of an MCP is really cool, but I kind of want to be 694 00:37:32,480 --> 00:37:34,600 over a background process. So maybe you know you had 695 00:37:34,600 --> 00:37:36,800 Cloudflare running in a loop having access to the Sentry MCP 696 00:37:36,800 --> 00:37:38,720 server and maybe if an error comes in it can do it. 697 00:37:38,720 --> 00:37:40,680 But I don't know. I got start thinking loops more 698 00:37:41,160 --> 00:37:41,560 well. I. 699 00:37:41,560 --> 00:37:45,400 Think actually self healing. I think we're all experimenting 700 00:37:45,400 --> 00:37:48,280 with that, right? Like this is I really shouldn't 701 00:37:48,360 --> 00:37:52,640 be, you know, trying to solve bugs manually like the system 702 00:37:52,640 --> 00:37:57,000 can know when they happen and heal and create APR like this 703 00:37:57,000 --> 00:37:59,320 should all be pretty automatic. So I think we're all realizing 704 00:37:59,320 --> 00:38:03,800 that, but it is messy and hard, isn't it to like actually get a 705 00:38:03,960 --> 00:38:08,320 a good self healing PR. There's a lot of false positives 706 00:38:08,320 --> 00:38:10,800 and it's interesting because they have a tool now and I 707 00:38:10,800 --> 00:38:14,200 forget the name of it, but it's, it's an assistant that you can 708 00:38:14,200 --> 00:38:17,520 fire off and it's like 20 bucks a month. 709 00:38:17,520 --> 00:38:20,280 And I was thinking maybe I'll try it, but I, I haven't pulled 710 00:38:20,280 --> 00:38:23,080 the trigger on it. And I, I probably just should, I 711 00:38:23,080 --> 00:38:25,600 think we're, we're all in this world now where you can build 712 00:38:25,600 --> 00:38:28,880 any software yourself now. And the question becomes, should 713 00:38:28,880 --> 00:38:33,160 I build it or should I buy it? And I, I think it's all up in 714 00:38:33,160 --> 00:38:34,720 there. Like people probably say, why in 715 00:38:34,720 --> 00:38:36,840 the world do you do you buy fizzy? 716 00:38:36,840 --> 00:38:39,840 That's hilariously dumb. Like it's just a Kanban board, 717 00:38:40,040 --> 00:38:43,840 but I'm like, I like it. I don't want to maintain that 718 00:38:43,840 --> 00:38:45,920 code, right, Even if it's the agent. 719 00:38:45,920 --> 00:38:48,920 So where do you think we're going with build versus buy on 720 00:38:48,920 --> 00:38:50,520 on dev tools? It's it's. 721 00:38:50,520 --> 00:38:55,600 Changing a lot because when and it also part of our problem, I'm 722 00:38:55,600 --> 00:38:57,760 assuming it's both of our problem is the joy of the build 723 00:38:57,840 --> 00:38:59,240 with it being this magical moment. 724 00:38:59,480 --> 00:39:01,320 It's like, of course I could do of course I can build my own 725 00:39:01,320 --> 00:39:05,760 e-mail drip management software and it's sometimes fun to just 726 00:39:05,760 --> 00:39:07,600 bring everything in house to get full control of the data. 727 00:39:07,840 --> 00:39:10,800 But yeah, one tool I want to give a little shout out to. 728 00:39:10,800 --> 00:39:14,080 Also not being paid but Greptile as APR review every part. 729 00:39:14,080 --> 00:39:18,160 Goes try that so I. Have a fleet, I've clawed, I 730 00:39:18,160 --> 00:39:21,760 have codex, I have cursor bug bot and Greptile running on all 731 00:39:21,760 --> 00:39:24,320 the PRS to go through and they're all good at their own 732 00:39:24,320 --> 00:39:26,880 different things. I would actually put Claude at 733 00:39:26,880 --> 00:39:28,480 the bottom of that list. Really. 734 00:39:28,480 --> 00:39:29,840 Even though I like Claude and other things. 735 00:39:30,120 --> 00:39:32,880 One of the issues is when you update APR with new changes, you 736 00:39:32,880 --> 00:39:35,320 solve some of the issues. Claude still takes the old code 737 00:39:35,320 --> 00:39:36,920 or the old comments into context. 738 00:39:36,920 --> 00:39:38,560 So it'll be like you have this critical issue. 739 00:39:38,720 --> 00:39:40,760 Oh wait, it's been solved. It's just it's, it's too much 740 00:39:40,760 --> 00:39:42,080 throwing me back and forth to like buddy. 741 00:39:42,440 --> 00:39:43,520 Come on. Yeah. 742 00:39:43,800 --> 00:39:46,600 Get us together. Cursor Bug Bot and Codex are 743 00:39:46,600 --> 00:39:49,000 both good. They find like good race 744 00:39:49,000 --> 00:39:50,720 conditions. They make connections to other 745 00:39:50,720 --> 00:39:52,720 parts of the code. But Reptile comes out with a 746 00:39:52,720 --> 00:39:55,760 very thorough analysis. There's a nice sequence diagram 747 00:39:55,760 --> 00:39:56,840 just so you can help visualize it. 748 00:39:56,920 --> 00:39:59,640 It keeps the human engaged with it, which I think is important 749 00:39:59,640 --> 00:40:02,840 because as we get more AI written code pushed up, I still 750 00:40:02,840 --> 00:40:04,760 want that human in the loop reviewing before it merges to 751 00:40:04,760 --> 00:40:07,840 master or merges to main. So I don't want to allow 752 00:40:07,880 --> 00:40:09,200 everything to be out the way, but yeah. 753 00:40:09,360 --> 00:40:11,440 Reptile Cool I've been thinking about it. 754 00:40:11,560 --> 00:40:16,240 I'm trying to pick my, my bug bottom choice because I'm not 755 00:40:16,240 --> 00:40:20,440 consistent enough, right? Like usually what I'll do is 756 00:40:20,640 --> 00:40:24,240 I'll this, this is the problem. Like sometimes I'm like, let's 757 00:40:24,240 --> 00:40:27,200 just, let's just work on main. I can't be bothered to, you 758 00:40:27,200 --> 00:40:29,880 know, even create a branch, which is bad. 759 00:40:30,000 --> 00:40:31,280 And it's because I'm the only deaf. 760 00:40:31,280 --> 00:40:34,120 Like I just, I can do whatever I want and untangle. 761 00:40:34,120 --> 00:40:35,560 I don't need to coordinate with anybody. 762 00:40:35,720 --> 00:40:39,560 But I think the discipline of creating a branch is often 763 00:40:39,560 --> 00:40:42,680 important because then you can do a code review, right? 764 00:40:42,680 --> 00:40:45,040 Whereas it's a little bit harder if you're just working on main. 765 00:40:45,360 --> 00:40:47,440 So I'm, I'm a little note to myself. 766 00:40:47,480 --> 00:40:53,360 OK, Ryan, work on branches more #1 #2 you know, pick up PR bot, 767 00:40:53,560 --> 00:40:57,440 like review bot, whatever it is like whether it's bug bot or 768 00:40:57,480 --> 00:40:59,840 Reptile, just do it. Because I was trying to hand 769 00:40:59,840 --> 00:41:02,960 crank it by like using the Oracle and AMP, which is great, 770 00:41:03,720 --> 00:41:06,600 but it's the, the process isn't baked in. 771 00:41:07,880 --> 00:41:09,640 So I mean, how much does Reptile cost? 772 00:41:09,640 --> 00:41:11,320 I presume it's like pretty nominal. 773 00:41:11,680 --> 00:41:12,760 I think it's the 20. Bucks a month. 774 00:41:12,800 --> 00:41:17,160 Classic, classic sass these days. 20 bucks for infinite 775 00:41:17,160 --> 00:41:20,280 knowledge all right. Man, it's yeah, just the the 776 00:41:20,280 --> 00:41:23,600 confidence it gives me and the time it saves versus, you know, 777 00:41:23,600 --> 00:41:25,040 fixing an issue with reaches production. 778 00:41:25,440 --> 00:41:27,440 Worth it for sure. And actually I've been 779 00:41:27,440 --> 00:41:30,320 experiencing something similar. I was the sole engineer at Box 1 780 00:41:30,320 --> 00:41:32,960 for four or five months, brought on Richardson lately who's been 781 00:41:32,960 --> 00:41:34,720 awesome. But the idea of going from 1:00 782 00:41:34,720 --> 00:41:37,160 to 2:00 is massively different engineering processes. 783 00:41:37,520 --> 00:41:38,920 So what are? You what are you encountering? 784 00:41:38,920 --> 00:41:41,000 Because I I'm going to encounter this. 785 00:41:41,000 --> 00:41:42,680 I'm curious. Tell teach me so I don't have to 786 00:41:42,800 --> 00:41:45,120 make the same mistakes. The big thing? 787 00:41:45,120 --> 00:41:46,480 Is just like baking of processes. 788 00:41:46,480 --> 00:41:48,280 Same thing for me. I'm like there's a bug fix just 789 00:41:48,280 --> 00:41:50,560 straight to main like who cares just send it. 790 00:41:50,560 --> 00:41:52,920 But now making sure there's like a proper review process. 791 00:41:54,080 --> 00:41:56,520 Database migrations. That's been an issue where 792 00:41:56,520 --> 00:41:57,720 working on things simultaneously. 793 00:41:57,720 --> 00:41:59,280 Both of us change the database, you go to merge. 794 00:41:59,280 --> 00:42:02,040 Oh now we have to make like a merge head onto. 795 00:42:02,160 --> 00:42:06,800 I use alembic through Python. It's been good, but as soon as 796 00:42:06,800 --> 00:42:09,480 you start running into migration issues, it's like, OK, before I 797 00:42:09,480 --> 00:42:11,640 just run it manual in my computer update production. 798 00:42:11,800 --> 00:42:14,280 Now I need to lock that down and make sure that we have it 799 00:42:14,320 --> 00:42:16,520 automatic in the CICD. So yeah, I would just say 800 00:42:16,880 --> 00:42:20,480 anytime you go against what you know is the best practice 801 00:42:20,480 --> 00:42:22,280 because it's it's simple, it's easy, it's convenient, you're a 802 00:42:22,280 --> 00:42:25,080 team of one. Try to put guard rails on it. 803 00:42:25,080 --> 00:42:28,240 Try to make sure that there is the option for multiple paths to 804 00:42:28,240 --> 00:42:30,920 converge safely. But it's one of those things 805 00:42:30,920 --> 00:42:33,000 growing pains that happens well and. 806 00:42:33,000 --> 00:42:36,880 Probably you're, you're setting yourself up for better teams of 807 00:42:36,880 --> 00:42:40,680 agents by doing that. And, and so I should be doing 808 00:42:40,680 --> 00:42:43,840 that anyway. It's interesting because like so 809 00:42:43,840 --> 00:42:47,000 in my past life, but so you know, start off as an individual 810 00:42:47,000 --> 00:42:49,440 web developer that that was fun and easy. 811 00:42:49,720 --> 00:42:51,960 And then I started companies and then I started hiring 812 00:42:51,960 --> 00:42:54,720 engineering teams and I got abstracted away from the 813 00:42:54,720 --> 00:42:58,760 machinery of shipping, right. And these are all things that 814 00:42:58,760 --> 00:43:01,960 like my VP Evange and my CTO would laugh and say, well, duh, 815 00:43:01,960 --> 00:43:05,360 Ryan, like, of course, these are the processes that of course you 816 00:43:05,360 --> 00:43:08,520 should have in place. But I think all of us now are 817 00:43:08,520 --> 00:43:11,680 sort of being put into, Oh, I'm acto now. 818 00:43:12,200 --> 00:43:16,160 Like I may be ACTO of one, but but pretty soon I'm going to be 819 00:43:16,160 --> 00:43:19,360 CTO with either humans or agents, you know, on my team and 820 00:43:19,360 --> 00:43:21,160 I I need to know how to orchestrate that. 821 00:43:21,440 --> 00:43:23,600 So it's kind of fun to be learning all these lessons. 822 00:43:23,600 --> 00:43:26,240 I wonder if it feels like they're is a company or a 823 00:43:26,240 --> 00:43:29,480 service here that somebody should launch, which is like a 824 00:43:29,480 --> 00:43:34,880 gentify, you know your process because that's important when 825 00:43:34,880 --> 00:43:36,640 you when you talk. About having like 3 clones of a 826 00:43:36,640 --> 00:43:40,880 code base operating on each of them, do you have any insight to 827 00:43:41,120 --> 00:43:44,040 like the orchestration aspect? Do you just manually contact 828 00:43:44,040 --> 00:43:46,120 switch from one to the next? Is there any, oh this one's 829 00:43:46,120 --> 00:43:47,960 done, give a little Ding or something? 830 00:43:49,320 --> 00:43:52,320 No, this is I think. I don't think any of us have 831 00:43:52,320 --> 00:43:56,080 really figured this out. It's, and what is the UI even 832 00:43:56,080 --> 00:43:59,120 here? Like I and we talked a lot about 833 00:43:59,120 --> 00:44:01,160 this at AMP and I know everyone's talking about it, but 834 00:44:01,160 --> 00:44:04,280 what, what is the, the UI of agent orchestration? 835 00:44:04,280 --> 00:44:05,520 Because that's where we're going. 836 00:44:06,680 --> 00:44:09,760 And I know the team is working with some exciting things and, 837 00:44:09,760 --> 00:44:12,680 and we'll see what happens. But no, it's pretty messy manual 838 00:44:12,680 --> 00:44:15,160 right now. And I honestly feel it's 839 00:44:15,320 --> 00:44:19,360 terribly unoptimized. You know, I, I posted a tweet 840 00:44:20,160 --> 00:44:22,120 where I was just like, I have massive FOMO now. 841 00:44:22,120 --> 00:44:26,560 Like I, I feel like if my agents aren't running 24/7, you know, 842 00:44:26,560 --> 00:44:31,640 I'm falling behind. And I think there's this real 843 00:44:32,520 --> 00:44:35,680 and it's real and it should be real fear that we all have now 844 00:44:35,680 --> 00:44:38,360 that everybody can ship so much faster. 845 00:44:38,520 --> 00:44:41,520 And you will fall behind and you and your Moat will disappear. 846 00:44:42,800 --> 00:44:44,720 But then it's like, OK, well, what are you shipping all the 847 00:44:44,720 --> 00:44:47,600 time? Like, and is it the right stuff 848 00:44:47,600 --> 00:44:51,240 even? And so there's this whole sort 849 00:44:51,240 --> 00:44:53,280 of, I don't know even know what to call it. 850 00:44:53,280 --> 00:44:57,760 It's it's sort of a a introspection into the actual 851 00:44:57,760 --> 00:45:00,960 process of building a company and a product and how fast you 852 00:45:00,960 --> 00:45:04,720 can actually move when humans are no longer factor. 853 00:45:06,800 --> 00:45:09,280 Maybe we just need a ICOS and we should just go out of the way. 854 00:45:09,680 --> 00:45:11,840 Yeah. Are you are you familiar with 855 00:45:11,840 --> 00:45:13,760 Boardy at all? No what's? 856 00:45:13,760 --> 00:45:17,800 That subordinates like. An AIVC where it's trying to 857 00:45:17,800 --> 00:45:19,440 source deals, audit deals, do all that for AC. 858 00:45:19,440 --> 00:45:22,080 So I mean, I feel like we have the AI from the engineering side 859 00:45:22,080 --> 00:45:24,480 coming in, the AI from the VC side coming in, and we still 860 00:45:24,480 --> 00:45:28,440 have this control in the middle. But I mean, besides taste and 861 00:45:28,600 --> 00:45:32,800 distribution, I can't think of too much else of remote these 862 00:45:32,800 --> 00:45:36,080 days. No, it, it is strange, you know, 863 00:45:36,240 --> 00:45:39,640 I do sort of wonder. And so if I had this loop that's 864 00:45:39,640 --> 00:45:42,560 saying, you know, here's one action item that you can do 865 00:45:42,560 --> 00:45:47,040 today to increase, you know, your sign up rate, you know, or 866 00:45:47,040 --> 00:45:50,400 one action item you can do to make the code, you know, 867 00:45:50,880 --> 00:45:55,440 stronger, harder, better. Like then what if I just set the 868 00:45:55,440 --> 00:45:58,080 agent to actually do all this? Like, would you end up actually 869 00:45:58,080 --> 00:46:01,800 with something good or would it just go into madness? 870 00:46:01,800 --> 00:46:07,000 Or, you know, I don't know. I think we'll find out funny 871 00:46:07,000 --> 00:46:10,000 experiments. To run, I do want to touch on 872 00:46:10,000 --> 00:46:15,080 before we before we wrap up, did you have any massive failures 873 00:46:15,200 --> 00:46:17,320 implanting one of these loops? Any lessons learned that you 874 00:46:17,320 --> 00:46:19,840 think someone who wants to like loopify their digital 875 00:46:19,840 --> 00:46:22,200 experience? Can you know, skip the failures 876 00:46:22,200 --> 00:46:24,440 to get to success faster? Yes. 877 00:46:24,800 --> 00:46:27,960 I think the biggest problems I had were when I didn't spend the 878 00:46:27,960 --> 00:46:31,840 time specifying the user stories and acceptance criteria. 879 00:46:32,040 --> 00:46:34,160 And I know we touched on that, but it's worth saying again it, 880 00:46:34,480 --> 00:46:36,360 it really is worth spending some time there. 881 00:46:37,480 --> 00:46:41,640 I think that's the the biggest failures I had by far. 882 00:46:42,400 --> 00:46:45,720 And a lot of those acceptance criteria, they came down to to 883 00:46:45,720 --> 00:46:50,120 browser testing when I, I didn't have the tooling set up for the 884 00:46:50,120 --> 00:46:52,240 agent to have the correct feedback loop. 885 00:46:52,560 --> 00:46:55,720 I would say you've got to set up good feedback loops and it's as 886 00:46:55,720 --> 00:46:58,840 simple as like using agent browser as you know your 887 00:46:58,840 --> 00:47:00,560 feedback loop for front end. Great. 888 00:47:01,040 --> 00:47:05,320 I think that's one thing. I think you know it when it 889 00:47:05,320 --> 00:47:09,320 does, there's a couple things with database migrations where, 890 00:47:10,440 --> 00:47:13,200 you know, it could get pretty gnarly pretty fast if you're not 891 00:47:13,200 --> 00:47:16,880 careful. So that's always scary, you 892 00:47:16,920 --> 00:47:20,560 know, And you know when you see sometimes that the agent wants 893 00:47:20,560 --> 00:47:24,160 to like run a migration on prod, you know, and you're like, what? 894 00:47:24,720 --> 00:47:29,440 You know, Oh my God, I swore I put that in my do not allow list 895 00:47:29,440 --> 00:47:34,000 like or, you know, I have a, you know, pre push hook like that 896 00:47:34,000 --> 00:47:37,480 should have stopped that like, but there's still these things 897 00:47:37,480 --> 00:47:40,080 that sometimes seem like they're about to happen that cause 898 00:47:40,400 --> 00:47:42,920 chaos. So those are a couple of the 899 00:47:42,920 --> 00:47:45,440 things I would encourage people to try this stuff. 900 00:47:46,000 --> 00:47:51,440 You know, the repo that is in my tweet article on my Twitter 901 00:47:51,440 --> 00:47:54,400 article is something you should be able to point your agent at 902 00:47:54,400 --> 00:47:56,320 and just say set up Ralph for me. 903 00:47:57,640 --> 00:48:03,200 And so I think that's a stark tank forward slash Ralph on GIFs 904 00:48:03,600 --> 00:48:08,400 and it's open source and and so try it and and see what you 905 00:48:08,400 --> 00:48:11,040 think. I think I think you're going to 906 00:48:11,040 --> 00:48:12,120 like it. Love that. 907 00:48:12,120 --> 00:48:14,040 Well. It cool thanks down below one 908 00:48:14,040 --> 00:48:17,000 note on the PRD, one technique I found that's been pretty helpful 909 00:48:17,000 --> 00:48:19,880 is instead of just having an AI assistant try to help you write 910 00:48:19,880 --> 00:48:21,520 it. Actually I have a just interview 911 00:48:21,520 --> 00:48:23,640 you where it asks you a series of questions knowing the 912 00:48:23,640 --> 00:48:25,440 context. So you come up with the answers 913 00:48:25,440 --> 00:48:27,120 and you have to put the thought process through it rather than 914 00:48:27,120 --> 00:48:28,960 just offloading that. But you're still getting the 915 00:48:29,400 --> 00:48:32,320 benefits of acknowledging blind spots and just thinking about 916 00:48:32,320 --> 00:48:33,720 things from different perspectives. 917 00:48:34,040 --> 00:48:37,520 But I do try to put the onus of the critical thinking on myself 918 00:48:37,520 --> 00:48:39,720 as much as possible. I got to keep this thing sharp. 919 00:48:39,720 --> 00:48:41,920 You know you. Have to use your brain sometime 920 00:48:41,920 --> 00:48:43,400 that's that's actually a good idea. 921 00:48:43,400 --> 00:48:46,920 I love that. Otherwise, you know, I think 922 00:48:46,920 --> 00:48:51,040 people should just try, try, try and use the tools. 923 00:48:51,800 --> 00:48:54,640 There's never a dumb question now because you have an an 924 00:48:54,640 --> 00:49:00,480 infinitely patient smart tutor to help you and, and just try. 925 00:49:01,280 --> 00:49:02,760 It's a lot of fun. Awesome. 926 00:49:03,080 --> 00:49:04,600 Ryan, it's been awesome talking to you again. 927 00:49:04,960 --> 00:49:07,120 How can people keep up with you? I'm on X. 928 00:49:07,120 --> 00:49:10,840 All the time. So just look for Ryan Carson. 929 00:49:10,840 --> 00:49:14,040 One word. I sometimes go to LinkedIn, but 930 00:49:14,040 --> 00:49:19,640 only when I when I hate my life. And so I try, I try to stay on X 931 00:49:20,040 --> 00:49:21,280 when I can. Thanks for listening to my. 932 00:49:21,280 --> 00:49:23,960 Conversation with Ryan Carson. I love having Ryan back on. 933 00:49:23,960 --> 00:49:26,160 It's great to talk to an entrepreneur who's also a 934 00:49:26,160 --> 00:49:28,600 builder, also experimenting with the latest and greatest. 935 00:49:28,840 --> 00:49:31,200 And it seemed like a perfect opportunity to talk about this 936 00:49:31,200 --> 00:49:33,960 loop that I kept hearing about that just hadn't given a fair 937 00:49:33,960 --> 00:49:36,120 shot yet. I hope you start thinking in 938 00:49:36,120 --> 00:49:39,440 loops, where can you apply AI to a process, not just an 939 00:49:39,440 --> 00:49:42,120 individual step, but have something continually looking, 940 00:49:42,120 --> 00:49:45,600 continually learning, continually evaluating that what 941 00:49:45,600 --> 00:49:47,440 it sees in order to create a better outcome. 942 00:49:47,880 --> 00:49:50,880 Don't just get it to create data points, get it to create 943 00:49:50,920 --> 00:49:54,240 insight, action, things that actually you can derive value 944 00:49:54,240 --> 00:49:55,520 from. I hoe you enjoyed this 945 00:49:55,520 --> 00:49:57,120 conversation and I'll see you next week.