1 00:00:06,840 --> 00:00:10,080 Well, everybody, I'm Shinobi Technical Editor at Bitcoin 2 00:00:10,080 --> 00:00:13,600 Magazine, joined by Peter Woola of Chain Code Labs. 3 00:00:14,560 --> 00:00:20,280 So we are here to discuss your latest obsession over the last 4 00:00:20,280 --> 00:00:22,680 few years. Cluster Manpool. 5 00:00:23,000 --> 00:00:30,720 Yep, I this is definitely not not the first time we've talked 6 00:00:30,720 --> 00:00:35,720 about this. I I hope it's among the last 7 00:00:35,720 --> 00:00:39,600 times I need to talk about it now that the changes have 8 00:00:39,600 --> 00:00:45,120 actually been merged in Bitcoin Core with plan to be in the 31 9 00:00:45,120 --> 00:00:49,960 dot O release later this year. I can imagine that's a familiar 10 00:00:49,960 --> 00:00:55,160 feeling with you at this point. It's been a long project. 11 00:00:55,160 --> 00:00:59,640 I mean, I've, I've had many big projects over over the past few 12 00:00:59,640 --> 00:01:04,040 years, but I mean, it's still a big one that it's good to have 13 00:01:05,560 --> 00:01:11,600 off my plate. So you you and Suha staff tour 14 00:01:11,600 --> 00:01:16,160 kind of like conceptualized this I think 3 years ago now roughly. 15 00:01:16,320 --> 00:01:19,640 Yeah, I think it was a discussion we had in the office 16 00:01:19,640 --> 00:01:30,600 here in February 2023 where, you know, we're thinking about all 17 00:01:30,600 --> 00:01:33,200 the problems that the current MAMPOL has. 18 00:01:33,200 --> 00:01:40,440 And in fact I think this was inspired by a talk Suez gave to 19 00:01:40,440 --> 00:01:44,440 us here internally just talking about the problems the MAMPOL 20 00:01:44,440 --> 00:01:48,440 has. And among them, it's not the 21 00:01:48,440 --> 00:01:55,280 only one, but I think it it sort of a nice demonstration of of 22 00:01:55,840 --> 00:02:00,040 how things are broken. And that's really the difference 23 00:02:00,040 --> 00:02:06,200 between how in the current mining algorithm we use one 24 00:02:06,200 --> 00:02:11,240 ordering and when deciding what transactions to remove from the 25 00:02:11,240 --> 00:02:13,800 mempu when it fills up and you don't have enough memory 26 00:02:14,040 --> 00:02:16,640 anymore. We we use a different ordering. 27 00:02:17,200 --> 00:02:22,680 And this was necessitated by the design of the code at the time, 28 00:02:23,440 --> 00:02:28,080 but it, it was still surprising. So in a bit more detail when, 29 00:02:28,080 --> 00:02:33,480 when jumping ahead, the the real problem is that you, you would 30 00:02:33,480 --> 00:02:37,920 hope that the mining algorithm picks the transaction in some 31 00:02:37,920 --> 00:02:41,800 order going from, you know, higher fee rate to lower fee 32 00:02:41,800 --> 00:02:44,400 rate order, but respecting topology. 33 00:02:44,600 --> 00:02:47,560 Because of course we can in Mampol have dependent 34 00:02:47,560 --> 00:02:52,560 transactions where you have, you know, I pay you in an 35 00:02:52,560 --> 00:02:56,120 unconfirmed transaction and then you use those coins and then 36 00:02:56,120 --> 00:02:58,560 spend them further to, to pay for something. 37 00:02:58,840 --> 00:03:03,440 So now we have a child transaction and there's logic in 38 00:03:03,440 --> 00:03:08,200 there so that if if your transaction pays a higher fee 39 00:03:08,200 --> 00:03:12,480 rate, these two will be considered together and we'll 40 00:03:12,480 --> 00:03:19,600 have that that's the child pays for parents concept. 41 00:03:20,120 --> 00:03:25,960 But so all of that block mining algorithm in Bitcoin Core today 42 00:03:25,960 --> 00:03:30,440 and as it has been since 2015 has taken all of this into 43 00:03:30,440 --> 00:03:32,240 account. So it roughly comes up with a 44 00:03:32,240 --> 00:03:36,320 good ordering, not perfect, we'll get into that later, but 45 00:03:37,480 --> 00:03:42,000 fairly good going from higher fee rate to lower fee rate with 46 00:03:42,000 --> 00:03:44,440 parents always before their children. 47 00:03:47,200 --> 00:03:53,960 And you would hope that when memory runs out, you start 48 00:03:53,960 --> 00:03:59,400 evicting transactions that would be considered last by this 49 00:03:59,400 --> 00:04:01,600 order. The very last, the first thing 50 00:04:01,600 --> 00:04:04,760 you want to evict is the last thing you would want to mine 51 00:04:04,760 --> 00:04:09,720 that this makes sense. And that's, that's true in most 52 00:04:09,720 --> 00:04:14,160 cases, but you can construct pathological examples of 53 00:04:14,840 --> 00:04:18,240 constellations of transactions within a member where this is 54 00:04:18,240 --> 00:04:21,279 not the case. And in, in very extreme cases, 55 00:04:21,519 --> 00:04:23,760 in fact, it, it appeared to be possible. 56 00:04:24,280 --> 00:04:27,440 She also discovered this, that the very first thing you evict 57 00:04:27,760 --> 00:04:31,800 is the last thing is the first thing you would, you would mine, 58 00:04:32,960 --> 00:04:36,680 which is obviously bonkers. That's not what we want to throw 59 00:04:36,680 --> 00:04:43,120 away. And I, I I I don't say that this 60 00:04:43,120 --> 00:04:46,920 is the only problem, or even the most important one. 61 00:04:47,720 --> 00:04:57,160 It just is a nice way of showing that what we really want inside 62 00:04:57,880 --> 00:05:04,440 is a a well defined single ordering on all transactions if. 63 00:05:04,560 --> 00:05:07,480 You want predictability. You want to know this is going 64 00:05:07,480 --> 00:05:09,520 to always happen in these conditions. 65 00:05:09,600 --> 00:05:11,960 Right. And and the the the the the the 66 00:05:11,960 --> 00:05:14,440 real problem. Why? 67 00:05:14,680 --> 00:05:17,120 Why are these orderings difference? 68 00:05:17,120 --> 00:05:21,480 Well, the the reason is, is roughly that in order to decide 69 00:05:21,480 --> 00:05:25,560 what you'd want to mine last, you basically need to run the 70 00:05:25,560 --> 00:05:31,720 mining algorithm for like a mem pool minus epsilon sized block 71 00:05:31,920 --> 00:05:34,200 and then see what you didn't mine. 72 00:05:34,400 --> 00:05:37,200 And well, if it that that that would work. 73 00:05:37,480 --> 00:05:40,840 But sadly this is computationally infeasible with 74 00:05:40,880 --> 00:05:46,960 all the transactions in there. So and and So what? 75 00:05:46,960 --> 00:05:49,160 What happens if you're faced with a problem? 76 00:05:49,160 --> 00:05:53,000 Well, we need a fixed ordering, but it is too computationally 77 00:05:53,000 --> 00:05:56,120 expensive to do. Well, what if we could pre 78 00:05:56,120 --> 00:06:00,120 compute things? What if instead of just running 79 00:06:00,120 --> 00:06:04,880 the mining algorithm at the time a book template is built? 80 00:06:04,880 --> 00:06:07,400 And that's only what a miner would do. 81 00:06:08,840 --> 00:06:14,320 We we in fact do it ahead of time whenever a change is made 82 00:06:14,320 --> 00:06:18,360 to to the memple, we we in fact keep a total ordering on all 83 00:06:18,360 --> 00:06:21,960 transactions in the memple at all times. 84 00:06:22,240 --> 00:06:25,920 And if we have this pre computed well then block building is 85 00:06:25,920 --> 00:06:28,280 easy. You pick from the front eviction 86 00:06:28,280 --> 00:06:32,000 is evict from the from the end. But there there are many more 87 00:06:32,000 --> 00:06:38,400 examples of parts of the Bitcoin Core codebase that in one way or 88 00:06:38,400 --> 00:06:43,240 another try to assess with a heuristic that's often in 89 00:06:43,840 --> 00:06:46,280 incorrect and often inconsistent. 90 00:06:47,400 --> 00:06:49,960 Is this transaction better than this one? 91 00:06:50,840 --> 00:06:54,640 Say there is a flood of transaction that come into your 92 00:06:54,640 --> 00:06:59,720 node as we've seen a year or two ago, or there have been such 93 00:06:59,720 --> 00:07:02,880 floods. There's a rate limiting process 94 00:07:02,880 --> 00:07:07,000 that will prevent that that flood from being propagated to 95 00:07:07,000 --> 00:07:10,360 all your peers and having them over be overwhelmed too. 96 00:07:10,760 --> 00:07:14,320 So you need to make a decision which transactions are you going 97 00:07:14,320 --> 00:07:17,280 to send first? Well, obviously the better one 98 00:07:17,280 --> 00:07:21,400 you want to send first, but it doesn't know that because we 99 00:07:21,400 --> 00:07:26,680 can't certainly at that time don't have computational need of 100 00:07:26,680 --> 00:07:29,840 time to go determine what things are. 101 00:07:29,840 --> 00:07:34,120 But there are even more examples like for fee rate estimation. 102 00:07:34,120 --> 00:07:38,480 It would be nice if fee rate estimation could take CPFP into 103 00:07:38,480 --> 00:07:41,200 account. It can't do that because it 104 00:07:41,200 --> 00:07:47,400 would need this information of adjusted goodness or effective 105 00:07:47,400 --> 00:07:50,000 fee rate as we call it, of transactions. 106 00:07:50,400 --> 00:07:56,200 And so all these problems we realized in fact that there's a 107 00:07:56,320 --> 00:08:00,840 whole bunch of ways that we're trying to compare quality of 108 00:08:00,840 --> 00:08:04,440 transactions with each other inside the code base in many 109 00:08:04,440 --> 00:08:08,480 different ways, all heuristically and all done 110 00:08:08,480 --> 00:08:12,160 independently. Another one is, is when fee 111 00:08:12,160 --> 00:08:14,920 bumping transactions in the wallets that there's, there's 112 00:08:15,440 --> 00:08:21,320 similar logic there and we really need to come up with a, a 113 00:08:21,320 --> 00:08:24,000 way to pre compute this total ordering. 114 00:08:24,000 --> 00:08:27,560 And if we have that and all these problems become so much 115 00:08:27,560 --> 00:08:31,720 simpler. The oh, the biggest one of all, 116 00:08:32,240 --> 00:08:35,520 replaced by fee. If you have a transaction that 117 00:08:35,520 --> 00:08:39,200 comes in that replaces another one, you need to know whether 118 00:08:39,200 --> 00:08:42,240 taking this new transaction and evicting all the things that 119 00:08:42,240 --> 00:08:44,640 conflicts with is actually an improvement. 120 00:08:45,640 --> 00:08:51,000 There's a bit 125 rules which are so sort of followed still, 121 00:08:51,000 --> 00:08:55,400 but they are heuristics and they are imperfect in many ways. 122 00:08:56,200 --> 00:09:01,240 There, there are ways today that you you can replace a 123 00:09:01,240 --> 00:09:05,200 transaction with one that is just unambiguously worse and 124 00:09:05,200 --> 00:09:10,160 gets accepted, and similarly many ways that you can have an 125 00:09:10,160 --> 00:09:14,000 unambiguously better transaction come in and will reject it for 126 00:09:14,000 --> 00:09:15,360 arbitrary reasons. Yeah. 127 00:09:15,360 --> 00:09:18,400 Just difference between like absolute fee and fee rates 128 00:09:18,400 --> 00:09:20,840 looking at size and how that's computed. 129 00:09:21,080 --> 00:09:24,600 So that is one that that's orthogonal so that the 130 00:09:24,600 --> 00:09:29,080 replacement rules, if we're going into that sort of boiled 131 00:09:29,080 --> 00:09:34,040 down into two categories, 1 is incentive compatibility in the 132 00:09:34,040 --> 00:09:38,160 sense of does taking this transaction make things better 133 00:09:38,160 --> 00:09:43,080 for the manpool, the minor in terms of fee income. 134 00:09:43,440 --> 00:09:46,800 And there's another set of rules which are about denial of 135 00:09:46,800 --> 00:09:50,880 service protection. So we, we want when a 136 00:09:50,880 --> 00:09:55,480 transaction replaces another one, we sort of charge the new 137 00:09:55,480 --> 00:10:02,240 transaction a fee for the cost of having relayed the old 138 00:10:02,240 --> 00:10:07,800 transaction, which will now not as we expected won't end up in a 139 00:10:07,800 --> 00:10:11,280 block anymore. And so that there is an absolute 140 00:10:11,280 --> 00:10:14,760 fee increase requirements that comes from there. 141 00:10:14,760 --> 00:10:16,280 Yeah, kind of just looking at bandwidth. 142 00:10:16,280 --> 00:10:20,520 Cost. Yes, this is just to prevent, 143 00:10:21,160 --> 00:10:25,080 you know, enabling avenues for an attacker to use the Bitcoin 144 00:10:25,240 --> 00:10:27,880 peer-to-peer network as a free relay mechanism. 145 00:10:28,360 --> 00:10:32,120 You're effectively at least imposing as much cost on them as 146 00:10:32,120 --> 00:10:35,320 relaying nodes is going to pay. So it's it's not like the nodes 147 00:10:35,320 --> 00:10:38,120 aren't being compensated, but they're still bearing the cost. 148 00:10:38,400 --> 00:10:40,840 Right. Ideally, we, you know, we we'd, 149 00:10:40,840 --> 00:10:46,560 we'd have the the transaction relayers receive the income that 150 00:10:47,160 --> 00:10:50,400 for relaying transactions, but that that really does. 151 00:10:50,400 --> 00:10:53,280 Figure that out, Let me know we can finally replace Bitcoin. 152 00:10:55,280 --> 00:10:59,560 I do believe there, there there was a very early research paper 153 00:10:59,560 --> 00:11:03,240 called the red balloons paper that tried to do this. 154 00:11:03,240 --> 00:11:06,360 But it's, it's not, I don't, I don't recall how it worked. 155 00:11:06,360 --> 00:11:09,920 And in any case, it's it's just not a practical problem because 156 00:11:10,560 --> 00:11:13,560 or, or rather, even if you have a solution for it, it's not a 157 00:11:13,560 --> 00:11:17,320 desirable outcome because it just means you will try to 158 00:11:17,320 --> 00:11:21,720 bypass the relay network that charges you a fee to relay and 159 00:11:21,720 --> 00:11:25,000 send it to minors directly, which is the very opposite of 160 00:11:25,000 --> 00:11:26,560 what we want to achieve. Yep. 161 00:11:26,800 --> 00:11:30,480 So replaced by fee rules, you have the incentive compatibility 162 00:11:30,480 --> 00:11:32,720 ones and you have the denial of service ones. 163 00:11:32,720 --> 00:11:36,840 We're not touching the denial of service protection rules, but 164 00:11:36,840 --> 00:11:41,480 the incentive compatibility ones today are wrong in both ways. 165 00:11:41,720 --> 00:11:45,560 Like they will accept things that are worse and they will 166 00:11:45,560 --> 00:11:47,280 reject many things that are better. 167 00:11:47,800 --> 00:11:53,360 And with sort of a framework of reasoning properly about how 168 00:11:53,360 --> 00:11:55,840 good transactions are with respect to each other. 169 00:11:56,000 --> 00:11:58,560 This involves a bit more than just ordering them. 170 00:11:59,800 --> 00:12:05,280 Like sort of come boils down to drawing a diagram of how the fee 171 00:12:05,280 --> 00:12:09,280 increases with the size and comparing those diagrams. 172 00:12:09,280 --> 00:12:13,880 But but still the cluster member framework in the end is is a 173 00:12:13,880 --> 00:12:16,880 solution to all all these problems at the same time. 174 00:12:17,400 --> 00:12:24,080 And so all right, we we want to impose a total ordering that we 175 00:12:24,080 --> 00:12:30,160 can pre compute, but sadly already plain like the we, we 176 00:12:30,160 --> 00:12:34,480 can't just run an imple sized version of the block building 177 00:12:34,480 --> 00:12:40,120 algorithm to determine the full ordering of everything at all 178 00:12:40,120 --> 00:12:45,000 times. And sadly, as things stand 179 00:12:45,000 --> 00:12:50,440 today, it it's in theory possible to relay a single 180 00:12:50,440 --> 00:12:55,240 transaction and completely change the optimal ordering of 181 00:12:55,280 --> 00:12:59,200 every transaction in the MPL, like completely reverse it. 182 00:12:59,320 --> 00:13:04,000 For example, we, we have an example like you have this chain 183 00:13:04,000 --> 00:13:06,880 of transactions sideways like parent, child, parent child, 184 00:13:06,880 --> 00:13:10,840 parents, child. And the ordering is like you 185 00:13:10,840 --> 00:13:12,480 first mind this, then this, then this. 186 00:13:12,480 --> 00:13:16,760 And now you add one very high fee rate transaction to the end. 187 00:13:16,760 --> 00:13:18,920 And suddenly it's this, this, this, this, this. 188 00:13:19,920 --> 00:13:23,960 So that's a problem because that means we need to deal with the 189 00:13:23,960 --> 00:13:27,600 case of the single transaction is being relayed to you, and you 190 00:13:27,600 --> 00:13:30,960 need to recompute the ordering of everything in your MAMPOL, 191 00:13:30,960 --> 00:13:34,240 however big it is, maybe hundreds of megabytes. 192 00:13:34,920 --> 00:13:38,720 And the solution to this is, well, what if we can just 193 00:13:38,720 --> 00:13:43,480 restrict how many transactions can be affected by any given new 194 00:13:43,480 --> 00:13:46,720 transaction coming in? And that's where the term 195 00:13:46,720 --> 00:13:49,880 cluster comes from. So we partition, the idea is to 196 00:13:49,920 --> 00:13:54,720 partition the manpool into groups of related transactions. 197 00:13:54,720 --> 00:13:58,960 And this includes parents, children, ancestors, 198 00:13:58,960 --> 00:14:03,040 descendants, but also descendants of your ancestors 199 00:14:03,040 --> 00:14:06,400 and their ancestors and their descendants and and and so 200 00:14:06,400 --> 00:14:09,680 forth. So anything that can be reached 201 00:14:10,240 --> 00:14:14,440 or any 2 transactions that that are related by an arbitrary 202 00:14:14,440 --> 00:14:19,880 combination of parent of child, of steps, they are considered to 203 00:14:19,880 --> 00:14:20,920 be in the same. It's just like. 204 00:14:21,120 --> 00:14:22,200 A A family tree? Yeah. 205 00:14:22,280 --> 00:14:25,680 Like visually? Yeah, like there's even the 206 00:14:25,680 --> 00:14:31,440 slightest relation between them that you, you, you think of them 207 00:14:31,440 --> 00:14:34,280 as the same. So it's it's your parents, your 208 00:14:34,280 --> 00:14:36,760 grandparents, your children, your grandchildren, but also 209 00:14:36,760 --> 00:14:41,240 your uncles, your nieces, your cousins, you're everything. 210 00:14:43,120 --> 00:14:48,480 And the idea is simple instead. So the the previous MAMPOL had 211 00:14:48,480 --> 00:14:53,840 some resource limitation rules. The default was that we would 212 00:14:53,840 --> 00:14:59,080 impose, and other things at most 25 ancestors including the 213 00:14:59,080 --> 00:15:03,480 transactions itself, and at most 25 descendants, including the 214 00:15:03,480 --> 00:15:07,400 transaction itself. And this is related directly to 215 00:15:07,440 --> 00:15:10,120 the computational cost of the mining algorithm and the 216 00:15:10,120 --> 00:15:14,320 eviction algorithm which needed to operate on these sets of 217 00:15:14,320 --> 00:15:16,320 ancestors and set of descendants. 218 00:15:16,320 --> 00:15:20,760 I won't go into the details here, but anyway. 219 00:15:20,760 --> 00:15:24,440 So with this new approach, we're replacing them those, those 220 00:15:24,440 --> 00:15:28,560 rules go away completely and they are replaced instead with a 221 00:15:28,560 --> 00:15:32,800 rule that clusters can be at most 64 transactions. 222 00:15:32,800 --> 00:15:36,440 The number is a bit bigger. So it, it does mean that you can 223 00:15:36,440 --> 00:15:40,640 build longer chains of transactions than 25. 224 00:15:42,680 --> 00:15:45,680 But of course it's it, it now works both ways. 225 00:15:46,080 --> 00:15:49,680 And so it's it's not, not. It's. 226 00:15:49,680 --> 00:15:52,560 It's not directionally bound, it's just total size. 227 00:15:53,520 --> 00:15:58,720 And your family can be at most 64 transactions rather than just 228 00:15:58,720 --> 00:16:00,840 your ancestors or just your descendants. 229 00:16:01,960 --> 00:16:05,640 Based we've we've done analysis on historical weight of the 230 00:16:05,640 --> 00:16:08,680 member, it doesn't appear that many transactions would be 231 00:16:08,680 --> 00:16:13,760 affected by this new rule. And of course, it will possibly 232 00:16:13,760 --> 00:16:17,040 enable new use cases that weren't possible before. 233 00:16:19,680 --> 00:16:24,640 And so with that, our solution is really partition them and 234 00:16:24,640 --> 00:16:28,000 pull into these clusters of at most 64. 235 00:16:28,000 --> 00:16:30,920 We impose a policy rule. You can't go above that. 236 00:16:30,920 --> 00:16:34,760 If you try to go above, we'll just reject the transaction. 237 00:16:36,360 --> 00:16:41,040 And whenever such a cluster is changed, a new transaction is 238 00:16:41,040 --> 00:16:43,760 added to it. Some part of it is mined, 239 00:16:43,960 --> 00:16:49,000 there's a reorg that conflict, some part out, there's a 240 00:16:49,000 --> 00:16:53,080 replacement that that that conflicts with some parts. 241 00:16:53,560 --> 00:16:58,600 Any change that's made to a cluster, we, we run effectively 242 00:16:58,600 --> 00:17:02,200 the block mining algorithm again, but just on that little 243 00:17:02,200 --> 00:17:04,520 cluster of at most 64 transactions. 244 00:17:04,520 --> 00:17:07,119 And because it's so small, this is fast. 245 00:17:09,000 --> 00:17:14,560 And it turns out that really all you need is this ordering within 246 00:17:14,560 --> 00:17:18,480 sets of related transaction, the the the overall ordering of them 247 00:17:18,480 --> 00:17:24,960 and pull this sort of a very simple merge sort of all your 248 00:17:24,960 --> 00:17:29,480 clusters and and this we can do at runtime whenever needed. 249 00:17:30,760 --> 00:17:35,640 But the the hard part, the computationally hard part is now 250 00:17:35,640 --> 00:17:40,440 restricted to just these groups of 64 transactions. 251 00:17:40,840 --> 00:17:45,640 Whenever a change is made, we just rerun A mining algorithm on 252 00:17:45,640 --> 00:17:47,280 it. And it's not a mining algorithm 253 00:17:47,280 --> 00:17:49,040 anymore. It's really just an algorithm 254 00:17:49,040 --> 00:17:52,760 for deciding the order of the transactions within it. 255 00:17:54,480 --> 00:17:59,440 Yeah. And of course what once we had 256 00:17:59,440 --> 00:18:04,640 that and we now actually do have that there. 257 00:18:04,640 --> 00:18:08,480 There is an obvious follow up question because, as I mentioned 258 00:18:08,480 --> 00:18:15,000 earlier, the existing logic for deciding transactions at block 259 00:18:15,000 --> 00:18:19,160 building time is suboptimal in many ways. 260 00:18:20,000 --> 00:18:24,680 This isn't necessarily a problem, but it's nice if we can 261 00:18:24,680 --> 00:18:28,440 do better and given that we're only going to be running this 262 00:18:28,440 --> 00:18:34,200 algorithm anymore on groups of 64 transactions, we may have the 263 00:18:34,200 --> 00:18:38,440 ability to do something far better than than what is done 264 00:18:38,440 --> 00:18:43,560 today. As an example, so today you have 265 00:18:44,920 --> 00:18:47,080 child pays for parent, which works. 266 00:18:47,400 --> 00:18:52,040 So you can have a single child that that pays for a parent and 267 00:18:52,040 --> 00:18:55,880 they will be aggregated and seen as a group, which is what you 268 00:18:55,880 --> 00:18:57,760 want. But what doesn't work is 269 00:18:57,760 --> 00:19:02,280 children pays for parent or children pay for parent grammar. 270 00:19:02,880 --> 00:19:06,840 So if you have two children that both have a higher fee rate than 271 00:19:06,840 --> 00:19:12,480 a parent, it only the the the one with the most effects will 272 00:19:12,480 --> 00:19:18,640 be considered alone rather than the combination of the two both 273 00:19:18,640 --> 00:19:23,560 bumping the parents together. I wouldn't say it's a problem 274 00:19:23,560 --> 00:19:25,480 because. It's a big blind spot. 275 00:19:25,800 --> 00:19:29,880 Yeah, it it's just I, I don't think it matters much in 276 00:19:29,880 --> 00:19:34,160 practice today because of course people aren't relying on use 277 00:19:34,160 --> 00:19:37,960 cases that need children pay for parent because it just doesn't 278 00:19:37,960 --> 00:19:43,440 work. But so this has has been my 279 00:19:43,440 --> 00:19:50,360 focus for I guess most of the past two years on and off some 280 00:19:50,360 --> 00:19:54,800 other things too, but is is try to come up with, you know, good 281 00:19:54,800 --> 00:19:59,240 algorithms for deciding. I have a group of at most 64 282 00:19:59,240 --> 00:20:02,640 transactions. What's the best ordering to to 283 00:20:02,640 --> 00:20:06,840 to mind them in? Hey everyone, it's Shinobi and 284 00:20:06,840 --> 00:20:09,360 I'm here in front of the Chicago Fed to talk to you about the 285 00:20:09,360 --> 00:20:12,720 core issue of Bitcoin Magazine. The last few years have been a 286 00:20:12,720 --> 00:20:16,000 bit of a communication breakdown between developers and people 287 00:20:16,000 --> 00:20:18,280 using Bitcoin. Bitcoin exists to be an 288 00:20:18,280 --> 00:20:20,000 alternative. To this institution. 289 00:20:20,360 --> 00:20:23,280 But to do that, it needs people to actively maintain it. 290 00:20:23,480 --> 00:20:26,320 If you actually want to hear from developers themselves, how 291 00:20:26,320 --> 00:20:29,360 they approach their work and what they choose to work on, go 292 00:20:29,360 --> 00:20:32,400 to bitcoinmagazine.com and get yourself a copy of The Core 293 00:20:32,400 --> 00:21:07,960 Issue. So it's pretty much like. 294 00:21:07,960 --> 00:21:10,880 You have this. Whole abstract architecture and 295 00:21:10,880 --> 00:21:14,920 new way of ordering things and managing the man pool, and now 296 00:21:14,920 --> 00:21:18,280 it's just, well, now what's the math of how we're going to 297 00:21:18,280 --> 00:21:20,280 manage? Yeah, it's just a drop in 298 00:21:20,280 --> 00:21:22,880 replacement, right. When we start this idea of 299 00:21:22,880 --> 00:21:26,240 cluster man pool, the idea was just to run the existing block 300 00:21:26,240 --> 00:21:32,040 mining algorithm on these now groups of 64 transactions ahead 301 00:21:32,040 --> 00:21:35,400 of time as opposed to do it on the whole manpool at blog 302 00:21:35,400 --> 00:21:39,640 building time. But given that we'll only be 303 00:21:39,640 --> 00:21:43,040 running in on groups of 64, maybe we can just do something 304 00:21:43,080 --> 00:21:46,720 better or specialize it. And in in. 305 00:21:46,720 --> 00:21:50,160 In practice we can basically find the optimal ordering 306 00:21:50,240 --> 00:21:53,880 always, which is neat. I mean, that's a little more 307 00:21:53,880 --> 00:21:56,360 than neat. I think that's the very good 308 00:21:56,360 --> 00:22:00,240 thing long term for the network to have like an optimal like 309 00:22:00,240 --> 00:22:03,760 most profitable algorithm like that to be open like anyone has 310 00:22:03,800 --> 00:22:06,760 access. To it anymore exactly because it 311 00:22:08,320 --> 00:22:11,920 before optimal. Our goal would be to be 312 00:22:11,920 --> 00:22:15,760 sufficient to cover all the things that people do on the 313 00:22:15,760 --> 00:22:18,480 network. Practically, if all that people 314 00:22:18,480 --> 00:22:24,480 do is single transaction CPFP bumping, then all you need is an 315 00:22:24,480 --> 00:22:26,640 algorithm that can deal well with that. 316 00:22:26,920 --> 00:22:32,000 But what if some use whose case, who, who, who knows what crazy, 317 00:22:32,440 --> 00:22:35,080 interesting or dumb things people come up with in the 318 00:22:35,080 --> 00:22:40,960 future that, you know, creates a strong economic incentive to pay 319 00:22:40,960 --> 00:22:45,160 for weird things in transactions? 320 00:22:45,440 --> 00:22:47,160 Well, if it's optimal, it's optimal. 321 00:22:47,360 --> 00:22:50,400 Like there, there, there's nothing people can come up with 322 00:22:51,080 --> 00:22:54,920 as long as it's restricted to 64 transactions and it's, it's 323 00:22:54,920 --> 00:22:58,680 still subject to the denial of service protection rules and so 324 00:22:58,680 --> 00:23:01,000 on. But you know, in in terms of 325 00:23:01,000 --> 00:23:04,400 topologies of related transaction, it doesn't matter 326 00:23:04,400 --> 00:23:09,840 what people come up with, we'll always order it the right way. 327 00:23:09,920 --> 00:23:12,480 Yeah. So it it's it's essentially 328 00:23:12,480 --> 00:23:16,240 future proof so that you can continue optimizing it based on 329 00:23:16,240 --> 00:23:20,800 like new second layers built new network behaviors or meta 330 00:23:20,800 --> 00:23:23,840 protocols or whatever people. You can come up with whatever 331 00:23:23,840 --> 00:23:30,040 use case and the OR the behavior of nodes just becomes well if 332 00:23:30,040 --> 00:23:33,480 it's better, I'll take it, if it's not better, I won't take 333 00:23:33,480 --> 00:23:39,800 it. And it it can discern better in 334 00:23:39,800 --> 00:23:44,480 US in any topology of of up to 64 transactions. 335 00:23:45,680 --> 00:23:48,640 I mean that. That sounds like a huge deal, 336 00:23:48,640 --> 00:23:53,800 especially for layer twos and just kind of giving more 337 00:23:54,680 --> 00:23:58,800 predictability or certainty as far as what people are building 338 00:23:58,800 --> 00:24:01,320 on top of. Especially for like reactive 339 00:24:01,320 --> 00:24:05,960 layer twos like Lightning or Arc where you might have to respond 340 00:24:05,960 --> 00:24:10,080 to another party's transactions and you need that guarantee. 341 00:24:10,080 --> 00:24:14,280 You need to know like if I submit this transaction like is 342 00:24:14,280 --> 00:24:17,680 this going to out compete something or replace something. 343 00:24:17,720 --> 00:24:23,760 It, it, it's true, but it, I wouldn't say it, it becomes more 344 00:24:23,760 --> 00:24:28,440 predictable in because it's predictable in the sense that 345 00:24:28,560 --> 00:24:34,480 it, it's at a very high level, super easy to give you what you 346 00:24:34,480 --> 00:24:37,680 need to do. You, you need to make it worth 347 00:24:38,200 --> 00:24:43,400 whatever you're doing needs to result in increasing fee income. 348 00:24:44,320 --> 00:24:46,200 And if you can do that, it will take it. 349 00:24:46,560 --> 00:24:52,960 But it it how that's decided. That's sort of a a very high 350 00:24:52,960 --> 00:24:58,440 level black box kind of thing where where before we had the 351 00:24:58,440 --> 00:25:02,440 big 125 rules which were not perfect and deviated from in in 352 00:25:02,440 --> 00:25:06,880 several ways, but they gave you a list of conditions like if you 353 00:25:06,880 --> 00:25:09,720 satisfy this rule, this rule, this rule, this rule, the 354 00:25:09,720 --> 00:25:12,840 replacement will go through. That's no longer the case. 355 00:25:12,840 --> 00:25:16,360 The rule is it needs to be better figure it out. 356 00:25:16,880 --> 00:25:21,960 I don't think this is an issue in practice because people 357 00:25:21,960 --> 00:25:29,080 really, if you're talking about groups of transactions where you 358 00:25:29,080 --> 00:25:33,880 or your protocol lightning or whatever layer to have, you 359 00:25:33,880 --> 00:25:37,720 know, control over all related transactions, that there is no 360 00:25:37,720 --> 00:25:41,360 issue Because you can of course, just run the algorithm on it and 361 00:25:41,360 --> 00:25:46,120 then see if it will be accepted. As soon as you interact with, 362 00:25:46,120 --> 00:25:52,000 you know, potentially grieving parties that might try to attach 363 00:25:52,000 --> 00:25:56,360 to your group of transaction. Like I, I make an unconfirmed 364 00:25:56,400 --> 00:26:01,680 output to a third party. Now they can attach arbitrary 365 00:26:01,680 --> 00:26:07,960 other things to that. And what I might need to do to 366 00:26:07,960 --> 00:26:13,960 bump a fee here becomes now dependent on what they're doing. 367 00:26:14,280 --> 00:26:18,400 So it it it does become, I think, harder to reason about, 368 00:26:18,400 --> 00:26:24,400 but it's but it's a far cleaner and it actually aligns with, you 369 00:26:24,400 --> 00:26:26,640 know, the incentive compatibility in the network 370 00:26:26,640 --> 00:26:31,040 where where before we had arbitrary rules that weren't 371 00:26:31,040 --> 00:26:34,800 actually always the best ones. And we shouldn't expect that 372 00:26:35,640 --> 00:26:39,400 going forward, those rules would be ones that miners keep 373 00:26:39,400 --> 00:26:44,080 following while this they hopefully will. 374 00:26:45,240 --> 00:26:48,120 I mean, I feel like, you know, this, at least from my 375 00:26:48,120 --> 00:26:53,280 perspective, it feels kind of like the latest point and like a 376 00:26:53,280 --> 00:26:57,280 progression of kind of cleaning up or removing incompatible 377 00:26:57,280 --> 00:27:00,480 things in the man pool. Like the original first scene 378 00:27:00,480 --> 00:27:03,400 safe rule. And then when we first 379 00:27:03,400 --> 00:27:07,800 implemented RBF, it was opt in. We had to flag it and then but 380 00:27:08,320 --> 00:27:11,360 miners were still mining on flag transactions anyway. 381 00:27:11,360 --> 00:27:14,520 And it's kind of been the, at least the entire time I've been 382 00:27:14,520 --> 00:27:18,600 in this space, people have been slowly cleaning up those things 383 00:27:18,600 --> 00:27:22,360 that people just did. Do you remember priority? 384 00:27:22,360 --> 00:27:23,920 I don't know how long. Ago where? 385 00:27:23,920 --> 00:27:26,560 You had the. The percentage of the block set 386 00:27:26,560 --> 00:27:29,480 aside for zero fee transactions if they were old enough. 387 00:27:29,840 --> 00:27:34,280 If they were, yeah, if they had sufficiently high bitcoins, they 388 00:27:34,280 --> 00:27:41,560 destroyed per byte, which I just made sense in in very early days 389 00:27:41,560 --> 00:27:46,280 because that was a useful prioritization mechanism. 390 00:27:46,280 --> 00:27:50,840 But of course I mean that that's just not maintainable in in. 391 00:27:51,560 --> 00:27:52,960 Yeah. Well, once the once there was 392 00:27:52,960 --> 00:27:56,600 real value and people competing for block space, no rational 393 00:27:56,600 --> 00:27:58,280 minor would would run that, Yeah. 394 00:27:59,040 --> 00:28:01,240 Exactly. So yeah, there, there. 395 00:28:01,240 --> 00:28:06,280 There's been a long sequence of steps in that direction, I 396 00:28:06,280 --> 00:28:08,600 think. I mean, it's a necessary thing 397 00:28:08,600 --> 00:28:12,200 in the long term, like if if we're going to use a system that 398 00:28:12,200 --> 00:28:15,880 depends on the incentives of these profit motivated players, 399 00:28:15,920 --> 00:28:19,280 like we should be aligned with their incentives or how do you 400 00:28:19,280 --> 00:28:21,280 think this is going to work? Exactly. 401 00:28:22,000 --> 00:28:26,160 You know, I think you gave a super stellar breakdown and I 402 00:28:26,160 --> 00:28:29,240 think most people watching might actually walk away understanding 403 00:28:29,240 --> 00:28:31,480 it. So I want to thank you a lot for 404 00:28:31,480 --> 00:28:33,400 that, Peter. Yes, Peter. 405 00:28:34,360 --> 00:28:35,920 And I hope you guys all learned something. 406 00:28:36,920 --> 00:28:37,280 By uh.