1 00:00:25,160 --> 00:00:39,040 The. Welcome to the Adamsport 2 00:00:39,040 --> 00:00:41,560 Territory Podcast. For this episode, I invited John 3 00:00:41,560 --> 00:00:43,400 Nixon to the show to talk us through the various 4 00:00:43,400 --> 00:00:46,840 considerations with recurring hardware for visa based VCF 5 00:00:46,840 --> 00:00:48,160 environment. Stewart. 6 00:00:48,520 --> 00:00:51,280 Welcome to the show, John. Howdy, happy to be here. 7 00:00:51,560 --> 00:00:53,760 Hey, this is a a a different one for the change. 8 00:00:53,760 --> 00:00:56,200 Normally I'm a guest on your podcast, but today you're a 9 00:00:56,200 --> 00:00:59,080 guest on my podcast. Now, for those who have never 10 00:00:59,080 --> 00:01:02,280 listened to virtual speaking, who is John Nicholson and what 11 00:01:02,280 --> 00:01:05,239 do you do for the company? You know, every time people say 12 00:01:05,239 --> 00:01:08,520 that I just have flashbacks to Office Space and being like I, I 13 00:01:08,520 --> 00:01:12,040 talk to customers for the engineers, but my name is John 14 00:01:12,040 --> 00:01:14,640 Nicholson. I work broadly in the storage 15 00:01:14,640 --> 00:01:18,080 space, primarily on VCN, but other cover other things. 16 00:01:18,120 --> 00:01:21,000 I've been in tech marketing, which if I'm not mistaken was 17 00:01:21,000 --> 00:01:22,680 Duncan's old job a long time ago. 18 00:01:22,680 --> 00:01:25,120 And you actually have a great blog, I think out there still on 19 00:01:25,120 --> 00:01:28,360 what is tech marketing. But I've owned the VCN design 20 00:01:28,360 --> 00:01:30,800 and sizing guide for the better part of a decade now. 21 00:01:30,920 --> 00:01:35,600 And I, you know, work on the how and why with VCN and new 22 00:01:35,600 --> 00:01:37,800 features and storage. Yeah, I think if you look at the 23 00:01:37,800 --> 00:01:41,200 content, there's typically, you know, only two people who get 24 00:01:41,400 --> 00:01:44,880 involved at this point of time. So it's either, of course, Pit 25 00:01:44,880 --> 00:01:47,080 Killer, who we had on the podcast multiple times, where 26 00:01:47,080 --> 00:01:48,880 it's yourself. Now, today I'm going to talk 27 00:01:48,880 --> 00:01:51,680 about something that is near and dear to your heart. 28 00:01:51,680 --> 00:01:54,520 I think some of the people may have seen some of your rants on 29 00:01:54,520 --> 00:01:57,360 Twitter about hardware configurations, what you should 30 00:01:57,360 --> 00:01:59,360 be buying, what you shouldn't be buying. 31 00:01:59,640 --> 00:02:01,840 And there are a couple of things that I just wanted to go over 32 00:02:01,840 --> 00:02:04,840 because I think, you know, some people have some serious issues 33 00:02:04,840 --> 00:02:07,560 when it comes to figuring out what their building material 34 00:02:07,560 --> 00:02:09,880 should look like. But before we actually dive into 35 00:02:09,880 --> 00:02:14,240 it, I gotta ask first, how do you feel about TPMS trusted 36 00:02:14,240 --> 00:02:16,000 platform modules? Because that's something that 37 00:02:16,000 --> 00:02:18,360 comes up regularly. It's, it's one of these things 38 00:02:18,360 --> 00:02:21,680 that, and I think this has spawned some rants, but it's 39 00:02:22,000 --> 00:02:26,120 what a TPM is there, there are kind of software TPMS in your, 40 00:02:26,160 --> 00:02:29,320 your desktop or your laptop type stuff, but hardware TPMS, which 41 00:02:29,320 --> 00:02:32,040 are commonly used in servers. It's this little like $50 thing 42 00:02:32,080 --> 00:02:35,400 and people forget them and it causes them unimaginable pain 43 00:02:35,400 --> 00:02:37,960 and suffering. And I, I think it's, it's a 44 00:02:37,960 --> 00:02:40,440 symptom of a larger thing that I think the rest of this episode 45 00:02:40,440 --> 00:02:43,360 will probably touch on is that I think a lot of people on 46 00:02:43,360 --> 00:02:46,480 procurement of servers, they've historically either, you know, 47 00:02:46,480 --> 00:02:49,560 in larger shops, they've had a procurement team or they've had 48 00:02:49,560 --> 00:02:52,200 a server OEM who's like, Hey, I've got this quick buy, you 49 00:02:52,200 --> 00:02:54,600 should buy it. And they don't think about the 50 00:02:54,600 --> 00:02:56,440 little details on that bill of materials. 51 00:02:56,440 --> 00:02:59,080 Their eyes just kind of glaze over when they see 12 items, 52 00:02:59,080 --> 00:03:03,360 including that TPM as an example, and it's very 53 00:03:04,360 --> 00:03:07,280 accidentally leaving something off or trying to save 50 bucks 54 00:03:07,280 --> 00:03:10,520 can cause you a lot of pain and suffering and and money. 55 00:03:11,040 --> 00:03:12,040 Yeah. And especially if you look at 56 00:03:12,040 --> 00:03:15,840 the configurations for the average server that we typically 57 00:03:15,840 --> 00:03:19,280 require right for V SAN or just for V Sphere in general, we're 58 00:03:19,280 --> 00:03:22,360 typically talking about thousands of dollars, closer to 59 00:03:22,360 --> 00:03:25,520 10s of thousands of dollars than then then thousand, so. 60 00:03:25,520 --> 00:03:27,760 Sorry Duncan, it's been a week. It's now 20,000 instead of a 61 00:03:27,760 --> 00:03:29,080 $10,000 server. So. 62 00:03:30,360 --> 00:03:32,400 Yeah, it's true. Yeah, the price of memory is 63 00:03:32,400 --> 00:03:35,560 going up fairly fast now. Besides TPM is one of the other 64 00:03:35,560 --> 00:03:38,320 things you also comment on fairly regularly, of course is 65 00:03:38,680 --> 00:03:42,560 network interface guards the Nic and I still see people for 66 00:03:42,560 --> 00:03:46,240 instance buying 10 gig NICs. So in your opinion, does that 67 00:03:46,400 --> 00:03:48,560 even make sense? What's your take on this SO? 68 00:03:48,800 --> 00:03:51,600 One thing to keep in mind with NICs is a lot of people they, 69 00:03:51,800 --> 00:03:54,760 they, they work backwards from, well, what, what switch am I 70 00:03:54,760 --> 00:03:57,360 connecting it to? Oh, I haven't, I have a, a nine 71 00:03:57,360 --> 00:04:01,160 year old 10 gig switch and I promised my CIO that I was going 72 00:04:01,160 --> 00:04:03,080 to get 10 years out of it. So I should just buy another 10 73 00:04:03,080 --> 00:04:05,360 gigs. I, you know, I have existing 74 00:04:05,360 --> 00:04:08,320 switch I'm going to plug into that or maybe I'm already using 75 00:04:08,320 --> 00:04:09,880 10 gig and I, I get happy with that. 76 00:04:09,880 --> 00:04:15,480 So there's a couple of things to think about. 1, you can use 25 77 00:04:15,480 --> 00:04:18,560 gig NICs with 10 gig. And if you look at the price 78 00:04:18,560 --> 00:04:23,080 difference between a 10 gig Nic and a 25 gig Nic, it's like $80 79 00:04:23,080 --> 00:04:25,600 or something. Or it's, it's, it's a rounding 80 00:04:25,600 --> 00:04:27,440 error. And that, that interface, 81 00:04:27,680 --> 00:04:32,040 assuming you're using optical or, or twin ax, that SFP plus 82 00:04:32,040 --> 00:04:37,160 that's 10 gig, those will work in a what's called SFP 28, which 83 00:04:37,160 --> 00:04:41,240 is the 25 gig standard. And so there is nothing stopping 84 00:04:41,240 --> 00:04:43,880 you if you have a 10 gig switch from just going ahead and 85 00:04:43,880 --> 00:04:44,800 buying. Because I see people getting 86 00:04:44,800 --> 00:04:48,520 this chicken egg problem where they say, well, because I have 87 00:04:48,520 --> 00:04:51,040 10 gig switches, I'm only going to buy 10 gig NICs. 88 00:04:51,040 --> 00:04:52,440 Well, I'm refreshing my switches. 89 00:04:52,440 --> 00:04:54,560 I'm going to buy 10 gig switches because I have 25 gig. 90 00:04:54,560 --> 00:04:56,640 And it's like you're, you're going to be stuck at 10 gig and 91 00:04:56,640 --> 00:04:59,080 you're just going to be circular buying for 30 years at this 92 00:04:59,080 --> 00:05:01,880 point. So we need to kind of snap out 93 00:05:01,880 --> 00:05:04,000 of that. We need to look at, OK, what is 94 00:05:04,000 --> 00:05:09,720 the cost increment if I go and buy, you know, a, a 25 gig top 95 00:05:09,720 --> 00:05:14,200 of rack data center class switch from, you know, an HP or a Dell, 96 00:05:14,200 --> 00:05:16,360 someone who's, you know, just getting commodity silicon, 97 00:05:16,360 --> 00:05:19,360 they're getting a probably Broadcom, all Hill Broadcom, you 98 00:05:19,360 --> 00:05:21,040 know, Trident 3X switch or something. 99 00:05:21,680 --> 00:05:24,240 And, and in many cases, what's actually funny is if you could 100 00:05:24,240 --> 00:05:26,280 try to buy a 10 gig switch, you're really actually buying. 101 00:05:26,280 --> 00:05:28,560 If you go look at the ASIC, it's actually that ASIC is actually a 102 00:05:28,560 --> 00:05:30,600 25 gig ASIC. You're just getting like a, a 103 00:05:30,600 --> 00:05:33,600 15% cost reduction for getting a 10 gig switch. 104 00:05:33,600 --> 00:05:36,240 It's, it's more of like AI won't call it a software limit. 105 00:05:36,240 --> 00:05:39,040 I'm sure there's some hardware limitations, but you're saving a 106 00:05:39,040 --> 00:05:42,920 little bit of money to have a significantly slower host. 107 00:05:43,200 --> 00:05:46,720 And you also need to think about what is the network impact in my 108 00:05:46,720 --> 00:05:49,120 environment? Well, historically when we were 109 00:05:49,120 --> 00:05:51,560 refreshing environments, we got bigger, our VMS get bigger. 110 00:05:51,560 --> 00:05:54,640 Over time, the amount of VMS that we have get bigger and we 111 00:05:54,640 --> 00:05:57,520 get bigger hosts and we do larger consolidation ratios or 112 00:05:57,520 --> 00:06:00,120 maybe we just cut our fleets in half if if our that 113 00:06:00,120 --> 00:06:03,960 application's not growing. Well, if you choke the network, 114 00:06:03,960 --> 00:06:06,680 there's a couple problems. So storage obviously near and 115 00:06:06,680 --> 00:06:09,560 dear to my heart, you're restricting your storage speeds, 116 00:06:09,560 --> 00:06:12,440 you're potentially having to deploy more nodes, you're 117 00:06:12,440 --> 00:06:14,360 potentially having application degradation. 118 00:06:14,360 --> 00:06:16,880 But also think about V motion. Think about V motion for a 119 00:06:16,880 --> 00:06:19,680 second here. How many you know, how quickly 120 00:06:19,680 --> 00:06:22,720 can I evacuate a host when I try to patch a host put into 121 00:06:22,720 --> 00:06:25,000 maintenance mode if I have 10 gig? 122 00:06:25,600 --> 00:06:27,520 When the 10 gig first came out, it was common. 123 00:06:27,520 --> 00:06:30,000 You might have had a host with 128 gigabytes of RAM. 124 00:06:30,240 --> 00:06:32,120 Now you're buying a host with a TB of RAM. 125 00:06:32,600 --> 00:06:34,600 If you're trying to evacuate that over the same 10 gig 126 00:06:34,600 --> 00:06:37,240 network, it's going to take 10 times as long. 127 00:06:38,160 --> 00:06:42,240 And you know, I know some of you, like myself, just Yolo OPS. 128 00:06:42,240 --> 00:06:44,760 You patch things, you hit, you know, upgrade everything and you 129 00:06:44,760 --> 00:06:46,880 wander off and go get eat a slop bowl for lunch. 130 00:06:47,360 --> 00:06:49,000 But there's some of you who actually have to be eyes on 131 00:06:49,000 --> 00:06:52,320 glass and do your patch cycles at night or you just want 132 00:06:52,320 --> 00:06:55,360 maintenance activities to not take, you know, forever. 133 00:06:56,320 --> 00:06:59,760 And for those that small upgrade and network performance going to 134 00:06:59,760 --> 00:07:02,000 25 or even going to hundreds, say, look, we have 10 times as 135 00:07:02,000 --> 00:07:03,320 much RAM. We should have 10 times as much 136 00:07:03,400 --> 00:07:06,320 emotion bandwidth that sorts of impacts and there's other 137 00:07:06,320 --> 00:07:09,240 decisions that people make backwards from that of I talked 138 00:07:09,240 --> 00:07:12,080 to a service provider said well we can't have a cluster larger 139 00:07:12,080 --> 00:07:15,360 than 12 nodes because we can't patch that in a 2 hour patch 140 00:07:15,360 --> 00:07:17,400 window. And that is a downstream 141 00:07:17,400 --> 00:07:20,040 decision of because they were still using 10 gig. 142 00:07:20,400 --> 00:07:23,120 It was, it was setting the maximum they're clusters. 143 00:07:23,120 --> 00:07:25,600 Like, you know, you ever have that conversation about the, 144 00:07:25,800 --> 00:07:28,680 there's this like saying about like the width of train tracks 145 00:07:28,680 --> 00:07:31,720 is just working backwards from roads and like the width of a 146 00:07:31,720 --> 00:07:35,240 horse or something in or wagons in Rome or something that we've 147 00:07:35,240 --> 00:07:38,160 just zombied on this, this tech debt or this architectural 148 00:07:38,160 --> 00:07:40,240 decision. That is what's happening with 149 00:07:40,240 --> 00:07:41,560 Nix. And you may say, well, I don't 150 00:07:41,560 --> 00:07:43,720 use that much bandwidth. Well, you know what, it's a 151 00:07:43,720 --> 00:07:45,800 marginal cost. It has second order effects. 152 00:07:45,800 --> 00:07:47,120 You might not need it in production, but during 153 00:07:47,120 --> 00:07:50,120 maintenance or other activities it's choking that and and the 154 00:07:50,120 --> 00:07:52,840 same thing with TPMS. You might not think you need it 155 00:07:52,840 --> 00:07:56,320 for secure boot today, but it now encrypts configs. 156 00:07:56,320 --> 00:07:58,320 It's now used for cashing V SAN keys. 157 00:07:58,320 --> 00:08:00,760 And when someone from compliance says, hey, banking regulations 158 00:08:00,760 --> 00:08:03,680 change, you have to do this. And then you have to figure out 159 00:08:03,840 --> 00:08:07,800 how do I reinstall the Sxi on 7000 hosts to because I save $50 160 00:08:07,800 --> 00:08:10,600 per host and I have to go one by one or the same with NICs, I 161 00:08:10,600 --> 00:08:13,320 have to go swap that. Think of that labor cost to 162 00:08:13,320 --> 00:08:15,760 that. So yeah, that was a lot of 163 00:08:15,760 --> 00:08:19,360 words, but there's a lot of downstream decisions of you 164 00:08:19,360 --> 00:08:22,320 saving money or just being on autopilot and saying, yeah, I 165 00:08:22,320 --> 00:08:23,440 just need the same thing as before. 166 00:08:23,640 --> 00:08:25,480 Yeah, I think that makes sense. And I actually had a 167 00:08:25,480 --> 00:08:28,800 conversation with a customer last week and they were talking 168 00:08:28,800 --> 00:08:31,240 about virtual machines with four terabytes of memory. 169 00:08:31,480 --> 00:08:34,799 So you can imagine, you know how long it would take to migrate 170 00:08:34,799 --> 00:08:36,960 those and those are active virtual machines. 171 00:08:36,960 --> 00:08:38,559 So they actually have a lot of active memory. 172 00:08:38,559 --> 00:08:40,919 So that would take a significant amount of time with 10 gigs. 173 00:08:40,919 --> 00:08:43,679 So I think those are very valid points. 174 00:08:43,679 --> 00:08:46,280 Now the other thing that you also already more or less 175 00:08:46,280 --> 00:08:48,760 alluded to and I think it's one of the things I also wanted to 176 00:08:48,760 --> 00:08:52,120 briefly discuss is the switch itself because it's one of those 177 00:08:52,120 --> 00:08:56,040 components from AV send perspective which has been often 178 00:08:56,040 --> 00:08:59,280 overlooked. And I think you've probably, you 179 00:08:59,280 --> 00:09:03,640 know, commented on this on our internal chat channels probably 180 00:09:03,640 --> 00:09:06,920 a billion times that people should more look into the 181 00:09:06,920 --> 00:09:09,960 network from a switching perspective because some of 182 00:09:09,960 --> 00:09:12,120 those switches may lack, you know, the correct buffers 183 00:09:12,120 --> 00:09:13,560 etcetera. So maybe you could talk us 184 00:09:13,560 --> 00:09:17,480 through some of those decisions around the the switches itself? 185 00:09:17,480 --> 00:09:20,440 So coming from the VM Ware world, I always joke I'm like 186 00:09:20,440 --> 00:09:22,720 the worst network engineer. I dropped out a net cab halfway 187 00:09:22,720 --> 00:09:25,560 through my CCNA. But as AVM wareness has said 188 00:09:25,560 --> 00:09:28,360 men, you should know what you're connecting to, even if that's a 189 00:09:28,360 --> 00:09:31,440 separate team. Because it's a bit like you're 190 00:09:31,440 --> 00:09:33,880 you're buying cars and you don't know what Rd. you're going to 191 00:09:33,880 --> 00:09:36,960 drive on or what highways those connect to. 192 00:09:37,040 --> 00:09:39,640 To extend this analogy further out from the top of rack to the 193 00:09:39,640 --> 00:09:41,840 leaf. So some terminology, you know, 194 00:09:41,840 --> 00:09:44,120 we're going to do some networking for for VM Ware 195 00:09:44,120 --> 00:09:48,000 admins here, modern networking, you basically have what's called 196 00:09:48,000 --> 00:09:50,280 leaf and spine topology. You have a leaf. 197 00:09:50,280 --> 00:09:52,200 This is what's sometimes called a Tor. 198 00:09:52,200 --> 00:09:54,640 Top of rack is what that is, is what your networking people are 199 00:09:54,640 --> 00:09:57,120 meeting there. You typically have two of these. 200 00:09:57,400 --> 00:09:59,480 Now these are generally going to connect back to something called 201 00:09:59,480 --> 00:10:02,640 a spine or several spines. Specifically you want redundant 202 00:10:02,640 --> 00:10:06,880 spines and some things to think about. 1 is the speed that you 203 00:10:06,880 --> 00:10:11,240 connect to and be aware that you can, just because the top of 204 00:10:11,240 --> 00:10:14,240 rack is a specific type doesn't mean that that actually forces 205 00:10:14,240 --> 00:10:16,280 your your hand on Nick's selection entirely. 206 00:10:16,520 --> 00:10:22,400 So like let's say your top of racks right now are, are 25 gig, 207 00:10:22,720 --> 00:10:25,800 but you know that you're going to refresh those in two years or 208 00:10:25,800 --> 00:10:27,160 maybe you just want to future proof yourself. 209 00:10:27,440 --> 00:10:30,640 You can actually go buy 100. There's some 100 gig NICs on the 210 00:10:30,640 --> 00:10:33,360 market you can buy today and you can get a breakout cable. 211 00:10:33,920 --> 00:10:36,240 And so that way down the road, if you ever do to site upgrade, 212 00:10:36,240 --> 00:10:38,080 you'll be ready. But you can actually get what's 213 00:10:38,080 --> 00:10:41,880 called a QSFP, which is what your 100 gigs typically run on Q 214 00:10:41,880 --> 00:10:44,080 stands for quad. You can get a four way break out 215 00:10:44,080 --> 00:10:48,600 to 420 fives on Broadcom on the single Nic parts Intel, the dual 216 00:10:48,600 --> 00:10:51,000 Nic parts, the 800 series do that Mellanox stone for some 217 00:10:51,000 --> 00:10:54,600 reason, I I believe, But you can also think about you can mix and 218 00:10:54,600 --> 00:10:56,720 these speeds can be and you can also go the other way. 219 00:10:56,720 --> 00:10:59,520 You could be, you might already have servers that are 25, but 220 00:10:59,520 --> 00:11:00,920 you're in the middle of a network refresh and you know 221 00:11:00,920 --> 00:11:03,000 what the marginal costs go to hundreds, not that much. 222 00:11:03,760 --> 00:11:07,120 You can go ahead and get 100 to 25 breakouts and be backward. 223 00:11:07,120 --> 00:11:08,960 You can be backwards before it's compatible. 224 00:11:09,400 --> 00:11:11,320 Now you asked about buffers and some of these other things to 225 00:11:11,320 --> 00:11:13,680 think about. One thing is do look at when you 226 00:11:13,680 --> 00:11:17,200 start running really heavy database workloads, you've got 227 00:11:17,200 --> 00:11:20,840 heavy write throughput. You've got lots of cluster to 228 00:11:20,840 --> 00:11:24,000 cluster chatter, which HCI by definition is that way, but also 229 00:11:24,000 --> 00:11:26,600 V motion is you can run into some of these problems called 230 00:11:26,600 --> 00:11:29,040 TCP in CAST where you have multiple ports trying to talk to 231 00:11:29,040 --> 00:11:31,960 the same port at the same time. And normally that would cause in 232 00:11:31,960 --> 00:11:35,280 TCP you would have a retransmit. If you want to Google something, 233 00:11:35,280 --> 00:11:37,440 Google TCP in cast. That's what this problem is 234 00:11:37,440 --> 00:11:39,440 known as. And some of the ways you can 235 00:11:39,440 --> 00:11:43,200 mitigate that, particularly for short bursts of traffic, often 236 00:11:43,200 --> 00:11:46,240 called microburst, which is how a lot of applications have is 237 00:11:46,240 --> 00:11:48,040 you can get deeper buffered switches. 238 00:11:48,320 --> 00:11:50,720 Now this is kind of come in and out of fashion with 10 gig for a 239 00:11:50,720 --> 00:11:54,360 while when they first came out, you know, 12 years ago, we just 240 00:11:54,360 --> 00:11:56,560 had shallow buffer. They were like 8 megabytes of 241 00:11:56,560 --> 00:11:58,440 buffer, which is not for the whole switch. 242 00:11:59,200 --> 00:12:01,360 And then people came out with, oh, let's come out with a 1 gig 243 00:12:01,360 --> 00:12:04,360 buffer. But then 25 gig shallow buffers 244 00:12:04,360 --> 00:12:06,320 came out. And you know, and the thing to 245 00:12:06,320 --> 00:12:09,320 think about is it's imagine you've got, you know, multiple, 246 00:12:09,440 --> 00:12:12,080 you know, bottles or cans on top of each other. 247 00:12:13,000 --> 00:12:15,120 And if you have that buffer, it's a place that you can store 248 00:12:15,120 --> 00:12:17,080 packets rather than them just falling on the ground or 249 00:12:17,080 --> 00:12:20,680 spilling over temporarily for that short burst and not having 250 00:12:20,680 --> 00:12:23,560 to pay that, that latency hit, that CPU hit that other things 251 00:12:23,560 --> 00:12:25,680 of having to go through the whole TCP retransmit cycle. 252 00:12:25,680 --> 00:12:28,400 So these ultra deep buffer switches when you hear that, 253 00:12:29,040 --> 00:12:32,080 that means you typically have a buffer in the multi gigabytes. 254 00:12:32,160 --> 00:12:35,600 And so if you've got bursty transactional workloads, things 255 00:12:35,600 --> 00:12:37,760 like that, these can help for your reliefs. 256 00:12:37,760 --> 00:12:42,040 Now your spines, you know, when we when we had multi layered, 257 00:12:42,040 --> 00:12:44,240 when we have a multi layer design, which we're going to 258 00:12:44,240 --> 00:12:47,240 have the thing to look for there is not actually buffers. 259 00:12:47,240 --> 00:12:49,760 The thing to look for there is the subscription ratio or 260 00:12:49,760 --> 00:12:54,960 oversubscription ratio. So if I have two switches at my 261 00:12:54,960 --> 00:13:01,280 top of rack that are 32 ports, 100 gig and I'm using sixteen of 262 00:13:01,280 --> 00:13:06,520 those for hosts, you know, then that and that leaves me 16 to 263 00:13:06,520 --> 00:13:08,040 connect to a spine that's a one to one. 264 00:13:08,040 --> 00:13:10,880 That means I'm not going to ever have contention between that 265 00:13:10,880 --> 00:13:14,080 leaf and that those spines that are aggregating that traffic 266 00:13:14,080 --> 00:13:17,360 together. Now if I oversubscribe that, 267 00:13:17,360 --> 00:13:20,120 maybe I'm using all but four of those ports and only have 4 268 00:13:20,120 --> 00:13:22,280 ports for that. Now I have an oversubscription 269 00:13:22,280 --> 00:13:23,680 ratio. Maybe I'm 4 to 1. 270 00:13:24,000 --> 00:13:26,320 And at 4:00 to 1:00, you got it when you start oversubscribing 271 00:13:26,320 --> 00:13:28,720 that leaf to spine. Now you need your networking 272 00:13:28,720 --> 00:13:31,040 team to do something that they they hate doing. 273 00:13:31,040 --> 00:13:34,440 You need them to monitor that. You need them to set alarms. 274 00:13:34,960 --> 00:13:38,120 And really what we're seeing in the industry as it used to be, 275 00:13:38,120 --> 00:13:40,440 you know, when I started 10 years ago, we said 3 to 1 was a 276 00:13:40,440 --> 00:13:42,880 good oversubscription ratio. Then we came two to one. 277 00:13:43,200 --> 00:13:46,120 When you went to your 48 port type switches with four uplinks 278 00:13:46,120 --> 00:13:48,440 and things like that, really what we're seeing is actually 279 00:13:48,440 --> 00:13:50,520 people going 1 to 1. And this sounds crazy because 280 00:13:50,520 --> 00:13:53,200 you're saying, look, half of my ports on my top of rack are 281 00:13:53,200 --> 00:13:55,360 going to be devoted to not servicing my house. 282 00:13:55,360 --> 00:13:58,320 They're going to a spine, but really what you're building 283 00:13:58,320 --> 00:14:01,520 there with that one to one model is you remember the chassis 284 00:14:01,520 --> 00:14:04,800 switches back in the day, like a 6509 or something really, you 285 00:14:04,800 --> 00:14:07,400 know, had those line cards and they had a backplane and they 286 00:14:07,400 --> 00:14:09,320 could all communicate. It all acted like one giant 287 00:14:09,320 --> 00:14:12,640 switch. What you're building by not 288 00:14:12,640 --> 00:14:15,000 oversubscribing is you're basically building a chassis 289 00:14:15,200 --> 00:14:16,720 that's distributed. And actually, by the way, 290 00:14:16,720 --> 00:14:19,080 chassis still exist and they're really still expensive. 291 00:14:19,440 --> 00:14:22,520 This leaf spine is actually cheaper, so you get the benefits 292 00:14:22,520 --> 00:14:24,040 of that. You also don't have a single 293 00:14:24,040 --> 00:14:26,640 point of failure 'cause that that backplane could fail. 294 00:14:26,640 --> 00:14:29,320 Very rare, but they could warp. Weird things could happen, but 295 00:14:29,320 --> 00:14:33,200 those multiple redundant spines can cover you and you can 296 00:14:33,200 --> 00:14:36,040 actually be in a situation where you no longer have to worry so 297 00:14:36,040 --> 00:14:38,440 much about monitoring. Your networking team can go off 298 00:14:38,440 --> 00:14:40,600 and solve other Wan problems or things like that. 299 00:14:40,640 --> 00:14:45,400 And one of our largest customers out there has written, they have 300 00:14:45,400 --> 00:14:46,720 a lot of blogs and things about this. 301 00:14:46,720 --> 00:14:48,280 They've written this and I've talked to them privately and 302 00:14:48,280 --> 00:14:50,920 they said it's great. Like the amount of overhead we 303 00:14:50,920 --> 00:14:53,520 had on monitoring and troubleshooting and, and ghosts 304 00:14:53,520 --> 00:14:55,600 in the machine and things like that just went away when we went 305 00:14:55,600 --> 00:14:58,000 to 1:00 to 1:00. And I encourage, you may say, 306 00:14:58,000 --> 00:14:59,920 wow, that sounds really expensive because if I'm buying, 307 00:14:59,920 --> 00:15:02,440 you know, 100 gig top of racks, I'm going to need 400 gig spine 308 00:15:02,440 --> 00:15:04,400 switches. But you start looking at the 309 00:15:04,400 --> 00:15:07,400 prices on these things, particularly when you're, you're 310 00:15:07,400 --> 00:15:09,960 not buying the most expensive proprietary ASICS, you're just 311 00:15:09,960 --> 00:15:12,760 buying, you know, the vendors like a rest and people use 312 00:15:12,920 --> 00:15:16,360 merchant silicon. If you compare that to the, all 313 00:15:16,360 --> 00:15:19,040 the money you're putting in to fill up a rack full of servers 314 00:15:19,040 --> 00:15:22,640 and RAM, it's maybe 15% of your bill image. 315 00:15:22,640 --> 00:15:25,280 It's not that much. So this is something I also 316 00:15:25,280 --> 00:15:28,640 encourage you when you look at cost, don't say, oh, wow, that 317 00:15:28,640 --> 00:15:30,680 spine switch is going to cost me $40,000. 318 00:15:31,120 --> 00:15:33,880 I want you to look at that and go, OK, that's $40,000, but 319 00:15:33,880 --> 00:15:35,800 that's servicing half a million in RAM. 320 00:15:36,160 --> 00:15:38,400 OK, that's probably worth it. Or, you know, maybe you are 321 00:15:38,400 --> 00:15:39,440 smaller. It's a different world. 322 00:15:39,440 --> 00:15:42,600 But look at these as a percentage of the total cost. 323 00:15:42,600 --> 00:15:46,080 And this is something how to think about is don't be that 324 00:15:46,080 --> 00:15:48,320 guy. Like remember that, you know, if 325 00:15:48,320 --> 00:15:50,920 you see somebody at the gym, the guy who skips leg day and has 326 00:15:50,920 --> 00:15:54,200 like giant arms, you know, and, and beefy stuff, don't be that 327 00:15:54,200 --> 00:15:56,120 guy in the data center who skips networking day. 328 00:15:56,200 --> 00:15:57,440 And I know we're not on the networking team. 329 00:15:57,440 --> 00:15:59,080 We're all VM Ware storage admins here. 330 00:15:59,080 --> 00:16:01,240 And we hate the networking team. They are the worst. 331 00:16:01,240 --> 00:16:04,200 It is correct. But we do need to actually 332 00:16:04,200 --> 00:16:06,440 encourage them to properly design and help, because 333 00:16:06,440 --> 00:16:08,640 otherwise we're going to have problems. 334 00:16:09,520 --> 00:16:12,680 I figured you were making a reference to Pete Fletcher when 335 00:16:12,680 --> 00:16:15,880 you were talking about the guy who skips leg day, but we're not 336 00:16:15,880 --> 00:16:17,400 going to go into that one. Now. 337 00:16:17,440 --> 00:16:21,120 The reason I actually invited you today was to specifically 338 00:16:21,120 --> 00:16:24,280 also talk about something which have been coming up more 339 00:16:24,280 --> 00:16:28,120 frequently and that is the emulated ready note and 340 00:16:28,120 --> 00:16:31,640 especially for V San ESA. Now before we start talking 341 00:16:31,640 --> 00:16:34,640 about some of the specifics, maybe you can explain what an 342 00:16:34,640 --> 00:16:38,360 emulated ready node is. So when ready, when we came out 343 00:16:38,360 --> 00:16:40,880 with ready nodes for ESA, we initially launched where they 344 00:16:40,880 --> 00:16:43,960 have to be ready nodes, they have to be these exact, they 345 00:16:43,960 --> 00:16:47,280 have to be purchased as a ready node BOM and things like that. 346 00:16:47,800 --> 00:16:50,640 And we were trying to, we did reduce a little bit of 347 00:16:50,640 --> 00:16:52,640 flexibility there, but we're also trying to be very 348 00:16:52,640 --> 00:16:55,560 prescriptive and avoid accidental ordering mistakes and 349 00:16:55,560 --> 00:16:58,440 things like that. So if you look, if you actually 350 00:16:58,440 --> 00:17:03,320 Google V SAN emulated ready node, you'll actually find a 351 00:17:03,320 --> 00:17:06,920 blog that was written by Pete Killer and he he goes through 352 00:17:06,920 --> 00:17:09,800 and talks about this that there is we have flexibility with the 353 00:17:09,800 --> 00:17:11,560 program. So let's say you already have 354 00:17:11,560 --> 00:17:14,400 some servers that meet the requirements of the ready node. 355 00:17:14,960 --> 00:17:19,000 You can use those so you don't have to go buy the chassis, you 356 00:17:19,000 --> 00:17:23,680 know explicitly that has the dash RN on under that code, you 357 00:17:23,680 --> 00:17:27,040 can buy an equivalent server that has the same, you know, 358 00:17:27,040 --> 00:17:30,320 backplane and Dr. connectivity. Now, one thing I will caution 359 00:17:30,320 --> 00:17:35,840 you on is note, you know a deal. When is the DL380 or 380? 360 00:17:36,280 --> 00:17:40,440 Well, the reality is the DL380 or an R650 or whatever server it 361 00:17:40,440 --> 00:17:42,640 is you're buying. You may see that model number 362 00:17:42,640 --> 00:17:45,760 and think they're all the same, but in reality it may be 1 of 12 363 00:17:45,760 --> 00:17:48,360 different completely different chassis codes in the back end 364 00:17:48,360 --> 00:17:51,120 that it has different, you know, maybe it has different drive 365 00:17:51,120 --> 00:17:54,840 sizes, It has 3 1/2 inch drives or maybe it has 2 1/2 inch, you 366 00:17:54,880 --> 00:17:58,600 know, universal form factor. Or maybe it uses the the you're 367 00:17:58,600 --> 00:18:02,240 one of the cool kids. You have the new EE3, the little 368 00:18:02,240 --> 00:18:04,040 ruler form factor drives that are really cool. 369 00:18:04,040 --> 00:18:06,440 You can go real dense on those are different. 370 00:18:06,440 --> 00:18:08,600 It's some of them. They make cable the drives to 371 00:18:08,600 --> 00:18:11,080 connect to a RAID controller, which on the V San world, we do 372 00:18:11,080 --> 00:18:14,120 not want. We do not like, please don't do 373 00:18:14,120 --> 00:18:16,960 that by by all means go buy RAID controllers for other use cases. 374 00:18:17,760 --> 00:18:20,240 You know, it's there's a probably Broadcom ASICS again, 375 00:18:20,320 --> 00:18:23,000 you know, Broadcom is all the things simultaneously. 376 00:18:23,000 --> 00:18:24,760 But this is the case where I'm going to tell you not to buy a 377 00:18:24,760 --> 00:18:27,360 Broadcom product. So we want to see Nvme direct 378 00:18:27,360 --> 00:18:29,320 connection and this stuff specified out there. 379 00:18:30,000 --> 00:18:32,600 But the other thing to, to pair with that, that guidance on 380 00:18:32,600 --> 00:18:35,640 emulated ready node of OK, I can I have a server, can I get it to 381 00:18:35,640 --> 00:18:37,080 look like a ready node, so to speak? 382 00:18:37,080 --> 00:18:41,000 Can I get certified drives? Is also look at there's a, 383 00:18:41,040 --> 00:18:43,680 there's a KB and there's a blog called what can I change on a 384 00:18:43,680 --> 00:18:45,560 ready node? And there are some things you 385 00:18:45,560 --> 00:18:47,800 can change. So people sometimes say, hey, 386 00:18:47,800 --> 00:18:49,960 look, the ready node has 22 cores. 387 00:18:49,960 --> 00:18:53,240 I want to buy the 24 quarts CPU. That is fine. 388 00:18:53,240 --> 00:18:56,160 You can absolutely change that. Do note, you know, changing the 389 00:18:56,160 --> 00:18:58,360 number of cores and frequency in them, it's going to have some 390 00:18:58,360 --> 00:19:00,200 performance impacts. They're, you know, and and 391 00:19:00,200 --> 00:19:04,480 again, don't skip leg day, don't don't go, you know, buy 128 392 00:19:04,480 --> 00:19:08,080 cores and have one drive and like 2 dims of RAM and you know, 393 00:19:08,640 --> 00:19:11,560 things like that. But there there are things you 394 00:19:11,560 --> 00:19:22,399 can change and that KB we actually step by That's a great. 395 00:19:22,400 --> 00:19:25,560 Idea. Now it's funny because that blog 396 00:19:25,560 --> 00:19:28,400 post has been out for a while and I barely heard anyone 397 00:19:28,400 --> 00:19:30,120 talking about it. But recently more and more 398 00:19:30,120 --> 00:19:34,400 people started talking about this specific concept. 399 00:19:34,400 --> 00:19:36,880 So why is that? Why all of a sudden is this 400 00:19:36,880 --> 00:19:39,640 relevant? So if you haven't gone to buy 401 00:19:39,640 --> 00:19:43,880 servers in the past three months, I'm sorry. 402 00:19:44,000 --> 00:19:45,920 It's going to, it's going to be fun. 403 00:19:47,760 --> 00:19:52,000 Historically, the price of, you know, memory and I feel like 404 00:19:52,000 --> 00:19:55,080 flash storage especially went down about 10% year over year. 405 00:19:56,000 --> 00:19:58,960 We're currently because of the AI super cycle and memory super 406 00:19:58,960 --> 00:20:01,600 cycle prices in some cases have quadrupled. 407 00:20:01,760 --> 00:20:03,920 And customers are taking a real hard look at procurement. 408 00:20:03,920 --> 00:20:08,400 There are a lot of customers who didn't try to right size, you 409 00:20:08,400 --> 00:20:10,680 know, they, they just bought more hardware whenever they 410 00:20:10,680 --> 00:20:12,520 needed something. And people are trying to say, 411 00:20:12,520 --> 00:20:15,200 OK, what do I have and how do I get the most value out of it? 412 00:20:15,560 --> 00:20:18,080 Because hardware prices are basically going in reverse. 413 00:20:18,080 --> 00:20:20,680 We're basically unspooling all the savings in the past several 414 00:20:20,680 --> 00:20:23,800 years. And this is, this is wild to the 415 00:20:23,800 --> 00:20:26,080 point to where, you know, I think kind of our original plan 416 00:20:26,080 --> 00:20:28,760 of record was sort of like, yeah, maybe we'll drop Cascade 417 00:20:28,760 --> 00:20:31,120 Lake from PCF 9. Now we're like, Oh yeah, yeah, 418 00:20:31,120 --> 00:20:32,920 we're, you're starting to see search show up on it. 419 00:20:33,240 --> 00:20:35,640 And I'm even hearing, you know, I think if you, there's some 420 00:20:35,640 --> 00:20:39,240 people even asking for RPQS on, on Skylake, which is wild 'cause 421 00:20:39,240 --> 00:20:41,320 it's like 6 generations ago CPUs. 422 00:20:41,680 --> 00:20:43,800 So people are trying to make do with what they have. 423 00:20:44,720 --> 00:20:46,680 Orders right now are incredibly painful. 424 00:20:46,680 --> 00:20:49,440 I mean, I've seen signed, executed, POS basically 425 00:20:49,440 --> 00:20:52,800 cancelled and the OEM come back and be like, hey, about that 426 00:20:52,800 --> 00:20:55,920 order, we're cancelling it and if you want it, we're just going 427 00:20:55,920 --> 00:20:58,000 to double it. I'm sorry, somebody else ordered 428 00:20:58,000 --> 00:21:02,440 $8 billion of, you know, AI hardware and that's not actually 429 00:21:02,440 --> 00:21:03,720 a joke. That's probably under selling 430 00:21:03,720 --> 00:21:05,360 how big that order was. Like this is the kind of 431 00:21:05,360 --> 00:21:08,040 situation we're in. And so some things people are 432 00:21:08,040 --> 00:21:11,240 looking at us saying, I've got some servers, I've got VCF 433 00:21:11,240 --> 00:21:13,200 licensing that has AVCN entitlement. 434 00:21:13,920 --> 00:21:16,640 How do I make, you know, lemonade from the, you know, 435 00:21:16,640 --> 00:21:18,800 sugar and lemons and ice that I have laying around. 436 00:21:19,200 --> 00:21:21,720 And that's, that's really driving a lot of these kind of 437 00:21:21,840 --> 00:21:24,120 discussions and that stuff. And it's, it's kind of fun 438 00:21:24,120 --> 00:21:27,160 because if you go back to the early days of virtualization, 439 00:21:27,160 --> 00:21:29,800 Duncan, you remember how you people really tries to squeeze 440 00:21:29,800 --> 00:21:31,880 the most out of VCP over subscription ratios. 441 00:21:31,880 --> 00:21:36,080 And they learned every best practice on DRS and, and memory, 442 00:21:36,080 --> 00:21:39,640 like we learned, you know, TPS and we learned about the balloon 443 00:21:39,640 --> 00:21:42,440 driver and we learned, you know, all the things you could do and 444 00:21:42,440 --> 00:21:45,080 VDI people would run 2 to one and things are three to one on 445 00:21:45,080 --> 00:21:48,080 RAM and crazy stuff. And, and we kind of got away 446 00:21:48,080 --> 00:21:50,400 from that because hardware got cheaper and we got lazy and we 447 00:21:50,400 --> 00:21:52,560 didn't want to deal with the occasional angry user who had an 448 00:21:52,560 --> 00:21:54,800 SLA violation. And I, I think we're kind of 449 00:21:54,800 --> 00:21:56,600 going backwards. We're getting back to the basics 450 00:21:56,600 --> 00:21:59,040 of how do we squeeze the most out of this hardware. 451 00:21:59,440 --> 00:22:01,840 And I mean, that's what VM Ware does. 452 00:22:01,840 --> 00:22:03,720 Well, that's our, that's our core competency. 453 00:22:04,600 --> 00:22:08,200 If people already own hardware, whether it's Dell, HPIBM, 454 00:22:08,200 --> 00:22:11,840 Lenovo, whatever it ends up being, and it could potentially 455 00:22:11,840 --> 00:22:15,360 be used for VSA and ESA, how would they go about validating 456 00:22:15,360 --> 00:22:16,560 that? How does that actually work? 457 00:22:16,560 --> 00:22:19,520 Because I've also unfortunately seen some people have already 458 00:22:19,520 --> 00:22:22,400 mentioned it, right? They try to create a 459 00:22:22,400 --> 00:22:24,640 configuration for themselves and then they ended up with this 460 00:22:24,640 --> 00:22:28,040 Dawn try mode controller and end up in a situation which is not 461 00:22:28,040 --> 00:22:29,680 supported. So how would they go about doing 462 00:22:29,680 --> 00:22:31,920 this? So some kind of quick like 463 00:22:31,960 --> 00:22:35,880 rules, rules of thumb that I see, for instance, if I see a 464 00:22:35,880 --> 00:22:39,480 RAID controller on that bill of materials, especially with Dell, 465 00:22:39,480 --> 00:22:43,040 like if I see PERC, I know that they've cabled all those drives 466 00:22:43,040 --> 00:22:44,760 to that. So I'm really going to I'm not 467 00:22:44,760 --> 00:22:46,720 going to be able to do ESA. I'm going to be doing OSA. 468 00:22:46,720 --> 00:22:48,240 Even if I have universal drive based. 469 00:22:48,960 --> 00:22:51,680 There are some configurations, some servers out there, like 470 00:22:51,680 --> 00:22:56,880 Cisco has specific sub chassis where if you have this code, the 471 00:22:56,920 --> 00:23:00,760 46NVM E drives got, you know, slaughtered a different 472 00:23:00,760 --> 00:23:03,560 direction or things like that. But for the most part, if you 473 00:23:03,560 --> 00:23:06,680 see a HP, a smart array, if you see a, a RAID controller on a 474 00:23:06,680 --> 00:23:09,600 Cisco, you see these things, that generally means, OK, this 475 00:23:09,600 --> 00:23:12,720 is going to be an OSA, This is cabled to where everything's 476 00:23:12,720 --> 00:23:14,160 going to connect to a central point. 477 00:23:14,240 --> 00:23:17,320 I don't have direct NVME. Now if you go back and you look 478 00:23:17,360 --> 00:23:19,360 and go back and find your original bill of materials. 479 00:23:19,720 --> 00:23:21,760 And if you find that and it says something, it's like NVME 480 00:23:21,760 --> 00:23:24,160 direct. OK, now we're, you know, we're, 481 00:23:24,200 --> 00:23:26,720 we're cooking. That means I've got PCIe raw 482 00:23:26,720 --> 00:23:29,200 lanes and, and some of the challenges, you may say, hey, 483 00:23:29,200 --> 00:23:31,800 look, we used to run OSA, you know, with drives, connect to 484 00:23:31,800 --> 00:23:33,080 the ray control. It wasn't that big of a deal. 485 00:23:33,080 --> 00:23:37,680 Well, some of the challenges are when you put NVME drives kind of 486 00:23:37,680 --> 00:23:40,880 RAID controller, some of them emulate them into SCSI drives, 487 00:23:40,880 --> 00:23:42,480 which is weird. And we, we, we just see 488 00:23:42,480 --> 00:23:45,680 performance regressions. The other fun thing is most Nvme 489 00:23:45,720 --> 00:23:47,800 drives are looking for four PCIe lanes. 490 00:23:47,800 --> 00:23:51,640 That's what they're expecting. And some of the, some of these 491 00:23:51,640 --> 00:23:54,080 RAID controllers, when you start to try to cable a lot of Nvme 492 00:23:54,120 --> 00:23:57,160 drives to them, somebody decided to save some money in the wiring 493 00:23:57,160 --> 00:23:58,920 harness or there were just limitations. 494 00:23:58,920 --> 00:24:01,840 And so they started just putting a single PCIe lane connection 495 00:24:02,280 --> 00:24:05,280 and the performance it's just it goes, it goes and then it just 496 00:24:05,280 --> 00:24:09,200 slams and so real. So and keep in mind this isn't 497 00:24:09,200 --> 00:24:11,680 just for ESA for OSA also, if you have one of those 498 00:24:11,680 --> 00:24:13,760 controllers, one, the controller has to be supported, but two, 499 00:24:13,760 --> 00:24:15,080 you're only going to be getting OSA. 500 00:24:15,160 --> 00:24:18,280 There is no ESA and there's no NVME. 501 00:24:18,280 --> 00:24:21,280 We don't support NVME behind Tri mode for OSA or ESA. 502 00:24:22,240 --> 00:24:24,720 Weird stuff happens. Maybe in the future someone 503 00:24:24,720 --> 00:24:26,760 might come out with the Tri mode controller doesn't have this 504 00:24:26,760 --> 00:24:30,120 problem, but you're also kind of contending with physics of that 505 00:24:30,120 --> 00:24:32,880 Tri mode controller is still sitting on an 8 or 16 PCIe 506 00:24:32,880 --> 00:24:34,880 lanes. And if you're trying to put 16 507 00:24:34,880 --> 00:24:37,680 drives on that, that's there's not enough lanes, it's 508 00:24:37,680 --> 00:24:39,880 problematic. There were some older 509 00:24:39,880 --> 00:24:42,720 configurations where people at PCIe switches sometimes referred 510 00:24:42,720 --> 00:24:45,680 to as a PLX and there are some of those on the already nodes. 511 00:24:45,680 --> 00:24:48,920 So you, you did have servers, particularly early Ice Lake and 512 00:24:48,920 --> 00:24:51,760 stuff where they would have this thing to get more in VME ports. 513 00:24:51,760 --> 00:24:55,320 But another thing to look for is you're looking for it with Intel 514 00:24:55,320 --> 00:24:57,600 Ice Lake or newer. Cascade Lake is not certified 515 00:24:57,600 --> 00:25:00,080 with ESA. There weren't enough Nvme lanes. 516 00:25:00,080 --> 00:25:01,920 It was Gen. 3 SSDs. They really weren't. 517 00:25:01,920 --> 00:25:04,200 The Nvme wasn't as fast. We were really wanting that Gen. 518 00:25:04,200 --> 00:25:06,680 4 and beyond capability. So we waited till Ice Lake. 519 00:25:07,440 --> 00:25:10,360 I'm guessing you're the Broadcom Compatibility guide would be 520 00:25:10,360 --> 00:25:11,960 your best friend at that point in time, right? 521 00:25:11,960 --> 00:25:14,800 Is that something that you would go over and over and over again? 522 00:25:14,840 --> 00:25:17,400 Yeah, so go to the Broadcom compatibility guide, find the 523 00:25:17,400 --> 00:25:20,360 matching ready node and look and pay attention as an OSA or an 524 00:25:20,400 --> 00:25:22,520 ESA. I talked to someone just today. 525 00:25:22,520 --> 00:25:24,120 It was like, oh, I got a bunch of eight 40s. 526 00:25:24,120 --> 00:25:26,800 I'm like, great, those are Cascade Lake, those are OSA and 527 00:25:26,800 --> 00:25:29,960 they're like, oh, I was like, well, you know is what it is. 528 00:25:31,120 --> 00:25:32,440 But these are things to look for. 529 00:25:32,440 --> 00:25:34,840 Look at the bill of materials. Compare your bill of materials 530 00:25:34,840 --> 00:25:36,400 against that. Pay attention to those 531 00:25:36,400 --> 00:25:38,960 controllers. Also, if you have really old 532 00:25:38,960 --> 00:25:42,040 hosts, you know, this is slightly off script here, but 533 00:25:42,120 --> 00:25:45,440 watch the boot device. If you've still got something 534 00:25:45,440 --> 00:25:48,760 booting from SD cards, you'll buy that little PC IE card with 535 00:25:48,760 --> 00:25:50,920 the m.two riser, the boss or whatever. 536 00:25:51,240 --> 00:25:54,600 You're just gonna have a happier life in general. 537 00:25:54,600 --> 00:25:57,800 And as we go forward, we need larger, we need larger endurance 538 00:25:57,800 --> 00:25:59,560 boot devices. They also boot faster. 539 00:25:59,680 --> 00:26:04,520 Like this isn't just a there's some nice to have to the also, 540 00:26:04,800 --> 00:26:06,320 this is the direction the world's moving. 541 00:26:06,920 --> 00:26:08,560 Yeah, I fully agree. It's something that actually 542 00:26:08,640 --> 00:26:11,840 does come up regularly still people using SD drives. 543 00:26:11,840 --> 00:26:14,760 Now the other thing that also comes up fairly frequently and 544 00:26:14,760 --> 00:26:17,360 something that of course people should be using the the Broadcom 545 00:26:17,360 --> 00:26:20,440 compatibility guide for are the NV me devices. 546 00:26:20,800 --> 00:26:23,120 Now one thing I've noticed lately and this seems to be a 547 00:26:23,120 --> 00:26:27,800 trend is that more customers are planning on procuring a lower 548 00:26:27,800 --> 00:26:31,080 number of devices with a higher amount of capacity. 549 00:26:31,280 --> 00:26:33,640 So what is your, what is your take or your opinion on that? 550 00:26:33,640 --> 00:26:37,240 Because you know, in my opinion, it, it's kind of a strange 551 00:26:37,240 --> 00:26:38,720 trend. So how do you feel about it? 552 00:26:38,720 --> 00:26:43,160 So there there's some things we've done to make it better, 553 00:26:43,160 --> 00:26:45,160 but there's still gotchas you got to watch out for. 554 00:26:45,240 --> 00:26:48,640 So one of these issues are if you, when you start going down 555 00:26:48,640 --> 00:26:51,400 to like one or two drives, you run into the situation to where 556 00:26:51,400 --> 00:26:54,640 if that drive fails, you basically could potentially be 557 00:26:54,640 --> 00:26:56,680 losing a fault domain. And that has like higher level 558 00:26:56,680 --> 00:26:58,280 rate issues. So I really like trying to get 559 00:26:58,280 --> 00:26:59,720 to at least you know, three drives or more. 560 00:27:00,120 --> 00:27:03,600 The other thing also is when you start going with real high small 561 00:27:03,600 --> 00:27:07,080 VM counts, there are some object limits that you're going to 562 00:27:07,080 --> 00:27:09,800 potentially run into if you're trying to run really, really 563 00:27:09,800 --> 00:27:13,320 dense, not a lot of capacity, but just you know, hundreds of 564 00:27:13,320 --> 00:27:17,400 tiny VMS on like sub six drives, you can get into some object 565 00:27:17,400 --> 00:27:19,600 limit considerations that you have to be aware of. 566 00:27:20,760 --> 00:27:24,920 The other kind of of gotcha with that is going to the opposite 567 00:27:24,920 --> 00:27:28,560 direction of just using a lot of small drives, you're potentially 568 00:27:28,560 --> 00:27:31,840 preventing yourself from having in place to capacity to expand. 569 00:27:31,840 --> 00:27:34,720 I think that's what drives is people say, you know, I want to 570 00:27:34,720 --> 00:27:38,120 only I want to fill in, you know, three of my 8 drives 571 00:27:38,120 --> 00:27:39,600 today. So that way I have lots of room 572 00:27:39,600 --> 00:27:42,600 for expansion. But you're potentially hitting 573 00:27:42,600 --> 00:27:45,360 either performance limits or object limits or you're, you 574 00:27:45,360 --> 00:27:47,560 know, potential fault doing limits if you go too small. 575 00:27:47,560 --> 00:27:49,520 So don't go to, don't go too high. 576 00:27:49,520 --> 00:27:53,400 Leave yourself a little room to expand or potentially leave room 577 00:27:53,400 --> 00:27:55,840 for memory tearing. You know, those also use drives 578 00:27:55,880 --> 00:27:58,360 that uses a drive slot or slots potentially. 579 00:27:59,080 --> 00:28:01,520 But the other thing here is, is, you know, don't undershoot this. 580 00:28:01,520 --> 00:28:04,720 Don't don't Again, it starts to look top heavy and weird when 581 00:28:04,720 --> 00:28:07,160 people try to run one or two or, you know, very few drives, 582 00:28:07,640 --> 00:28:08,920 especially in denser environments. 583 00:28:08,920 --> 00:28:12,400 And I know we've got, you know, dedupe and things like that and 584 00:28:12,400 --> 00:28:14,680 compression and we've gotten more efficient on this, but this 585 00:28:14,680 --> 00:28:17,120 is something to watch out for. And I, I will say the Gen. 5 586 00:28:17,120 --> 00:28:19,440 drives are pretty fast. So it used to be, if we go back 587 00:28:19,440 --> 00:28:22,440 to the early days of Esan, you remember we needed to use all 24 588 00:28:22,440 --> 00:28:24,320 drive slots. It felt like to get the most 589 00:28:24,320 --> 00:28:26,560 performance. You, you don't need to run 24 590 00:28:26,560 --> 00:28:28,680 drives to hit the, you know, you're going to the, the you'll, 591 00:28:28,880 --> 00:28:31,680 you'll even with all the threads that ESA can go, you're 592 00:28:31,680 --> 00:28:35,360 probably, you know, going to hit that, you know, 300,000 IOPS, 593 00:28:35,760 --> 00:28:38,880 you know, whatever per node wall it'll be based on the CPU. 594 00:28:39,120 --> 00:28:41,600 You'll probably hit that long before you can saturate 24 595 00:28:41,600 --> 00:28:44,400 drives where the capability. But there is there is kind of a 596 00:28:44,400 --> 00:28:46,720 sweet spot of trying. I try to shoot for at least 6 597 00:28:47,280 --> 00:28:49,280 and one thing to look at is if you think, well I've, you know, 598 00:28:49,280 --> 00:28:51,840 I'm using one U pizza boxes, I only have a drive base. 599 00:28:52,280 --> 00:28:56,160 Look at that E3 form factor because you can get 12 or 16 600 00:28:56,160 --> 00:28:58,240 drives to those in one U and at that point you really have no 601 00:28:58,240 --> 00:29:00,680 excuse not to. You're still leaving yourself 602 00:29:00,680 --> 00:29:04,160 room to grow without going too low on those minimums. 603 00:29:04,600 --> 00:29:07,720 Oh, it's funny, you mentioned the component limit on a per 604 00:29:07,720 --> 00:29:10,240 device basis. It is something that has never 605 00:29:10,240 --> 00:29:14,240 come up with with V San in the past two years, but in the last 606 00:29:14,240 --> 00:29:17,280 three weeks we had two customers actually complaining about this. 607 00:29:17,280 --> 00:29:19,400 So I think it's a very valid point to take that into 608 00:29:19,400 --> 00:29:21,520 consideration. Now the other thing when it 609 00:29:21,520 --> 00:29:24,760 comes to NVMES, which is also something that probably people 610 00:29:24,760 --> 00:29:27,200 should be taking into consideration are the types of 611 00:29:27,200 --> 00:29:29,840 device that they use. And of course, these days we see 612 00:29:29,840 --> 00:29:35,040 read intensive, write intensive, mixed and we support basically, 613 00:29:35,160 --> 00:29:37,840 you know, fairly large variety of drives. 614 00:29:38,320 --> 00:29:40,840 Some of those who have, you know, one drive right per day, 615 00:29:41,320 --> 00:29:44,360 some of them more. What's the consideration when it 616 00:29:44,360 --> 00:29:46,280 comes to buying those types of devices, and what do you 617 00:29:46,280 --> 00:29:47,680 typically recommend for customers? 618 00:29:47,680 --> 00:29:50,840 So this was fun when we first came with the ESA, we just 619 00:29:50,840 --> 00:29:52,760 supported the three drive right per day mixed-use. 620 00:29:54,000 --> 00:29:56,720 And then we we added read and what are called read intensive 621 00:29:56,720 --> 00:30:00,000 drives and read intensive. It's a weird name because it's 622 00:30:00,000 --> 00:30:03,720 it's like it's it's like saying my tires are parking optimized. 623 00:30:03,720 --> 00:30:06,200 It's not that they're better at reads than the write intensive 624 00:30:06,200 --> 00:30:07,920 drives. It just means they're not as 625 00:30:07,920 --> 00:30:10,280 good at writes. They don't have as much 626 00:30:10,280 --> 00:30:11,760 endurance. They have a third the endurance 627 00:30:11,800 --> 00:30:13,600 and and let's kind of unpack why. 628 00:30:13,600 --> 00:30:15,760 Like let's look under the hood. So you'll notice these these 629 00:30:15,760 --> 00:30:21,640 sizing increments of like 1.6 and 3.2 for the mixed-use, the 630 00:30:21,640 --> 00:30:23,480 three drive write per day. And then you'll see right next 631 00:30:23,480 --> 00:30:27,560 to it a 1.94 and a 3.843 point, whatever that is. 632 00:30:29,240 --> 00:30:31,880 And it's about a 2022% difference in capacity. 633 00:30:31,960 --> 00:30:33,600 And it's like, huh, that's funny. 634 00:30:33,600 --> 00:30:35,000 And I notice it's like this lock step. 635 00:30:35,360 --> 00:30:38,440 So what it is, is where those, those extra two drive rights per 636 00:30:38,440 --> 00:30:42,360 day is coming is that drive is basically they're functionally 637 00:30:42,360 --> 00:30:45,880 in many cases the exact same Dr. effectively it's same controller 638 00:30:45,880 --> 00:30:49,240 or very similar controller. It's it's the same NAND. 639 00:30:49,240 --> 00:30:52,160 It's the same actual flash die underneath, but they're actually 640 00:30:52,160 --> 00:30:55,440 just reserving about 2022% of that capacity, keeping it in 641 00:30:55,440 --> 00:30:57,480 reserve. So as cells die, then they can 642 00:30:57,480 --> 00:31:00,080 reallocate and rebalance things. So that's actually where the 643 00:31:00,080 --> 00:31:02,880 endurance have no one as far as more has come out with a 644 00:31:02,880 --> 00:31:04,760 firmware to like re flash it one way or the other. 645 00:31:04,760 --> 00:31:08,480 But that's that's kind of what that is the and and they're 646 00:31:08,480 --> 00:31:11,480 priced often about the same. So it's about a 2022% price 647 00:31:11,480 --> 00:31:12,720 difference. So people say, hey, cheaper 648 00:31:12,720 --> 00:31:17,800 capacity, let's go this way. Now, realistically, most people 649 00:31:17,800 --> 00:31:21,160 are not that right heavy and if you are, you probably should 650 00:31:21,160 --> 00:31:22,760 know who you are. You've got some VRA environment 651 00:31:22,760 --> 00:31:24,920 from hell that's repaving 500 times over. 652 00:31:24,920 --> 00:31:27,840 You're doing full clone VDI and you're doing like 100 cycles a 653 00:31:27,840 --> 00:31:30,600 day. Go look at your change reports 654 00:31:30,600 --> 00:31:32,840 from backups. Go look at your in OPS at your 655 00:31:32,840 --> 00:31:34,240 rights. You know if it if you're seeing 656 00:31:34,240 --> 00:31:36,160 really crazy stuff that makes you there. 657 00:31:36,160 --> 00:31:38,520 The other one is you're trying to get the highest consistent 658 00:31:38,520 --> 00:31:41,440 performance, particularly on high sustained rights. 659 00:31:41,760 --> 00:31:43,720 Those mixed-use drives are still going to perform better. 660 00:31:44,720 --> 00:31:47,400 Also the mixed-use are probably going to be you. 661 00:31:47,600 --> 00:31:49,600 Even if you're going to read intensive, you may still want to 662 00:31:49,600 --> 00:31:51,880 put one or two of those mixed-use in there for memory 663 00:31:51,880 --> 00:31:54,520 tearing because memory tearing is going to be a little more 664 00:31:54,520 --> 00:31:57,280 sensitive on performance. And that may, it's probably 665 00:31:57,280 --> 00:31:59,160 worth it, especially because you're not really using that 666 00:31:59,160 --> 00:32:01,040 extra capacity in many cases either way. 667 00:32:01,480 --> 00:32:05,680 So speaking to people who work in the industry on the flash 668 00:32:05,680 --> 00:32:08,920 side, 80% of the market in aggregate, not just the same, 669 00:32:08,920 --> 00:32:11,320 but in aggregate is kind of moved to read the read intensive 670 00:32:11,320 --> 00:32:13,400 drives. They are becoming kind of the de 671 00:32:13,400 --> 00:32:17,480 facto standard, but for high endurance environments, memory 672 00:32:17,480 --> 00:32:22,640 cheering use cases, stuff that's just really, really you're not 673 00:32:22,760 --> 00:32:25,160 as focused on capacity costs, you're really focused on right 674 00:32:25,160 --> 00:32:26,920 latency consistency under heavy load. 675 00:32:27,200 --> 00:32:29,040 You're still going to go for that mixed-use drive. 676 00:32:30,120 --> 00:32:32,480 But the other thing also you're going to, you're going to see is 677 00:32:32,480 --> 00:32:34,680 we can talk about all this in theory and then you're going to 678 00:32:34,680 --> 00:32:36,640 go talk to your distributor, you're going to talk to your OEM 679 00:32:36,640 --> 00:32:38,360 because that's no matter what they have in stock. 680 00:32:38,480 --> 00:32:42,760 And so I'm seeing some of the purchasing decisions being made 681 00:32:42,760 --> 00:32:46,680 based off of what's available as as prices are going up, as 682 00:32:46,680 --> 00:32:49,840 supply is getting crunched, people are often just having to 683 00:32:49,840 --> 00:32:51,480 make do. And this is also waking people 684 00:32:51,480 --> 00:32:54,800 up to decisions, which is good. But this is another thing to 685 00:32:54,800 --> 00:32:56,320 consider. And actually one of the most 686 00:32:56,360 --> 00:32:59,840 crazy like second order effects of purchasing decisions I ran 687 00:32:59,840 --> 00:33:03,400 into recently was someone was doing a very large order and 688 00:33:03,400 --> 00:33:07,160 they went to they wanted 256 gigabyte Dems for RAM and 689 00:33:07,440 --> 00:33:08,880 they're like, great, we're going to buy all this RAM. 690 00:33:09,680 --> 00:33:12,520 And they came back and their OE Ms. like, yeah, we can't get 691 00:33:12,520 --> 00:33:13,560 those to you at a reasonable cost. 692 00:33:13,560 --> 00:33:15,680 All those Dems are going to the AAI weirdos. 693 00:33:16,400 --> 00:33:19,600 But you know, we can just sell you twice as many 120 eights and 694 00:33:19,600 --> 00:33:21,240 they go, OK, we'll do twice as many 120. 695 00:33:21,240 --> 00:33:25,280 Wait, that's a lot more power. We're maxed on power. 696 00:33:25,640 --> 00:33:29,640 OK, can we double the size of our V SAN drives and cut them in 697 00:33:29,640 --> 00:33:34,560 half so we can save power to pay for the extra the more Dems so 698 00:33:34,560 --> 00:33:37,080 we don't have to go to deploy more generators and more cooling 699 00:33:37,080 --> 00:33:40,400 in this data set? Like this is the kind of like, 700 00:33:40,520 --> 00:33:43,760 you know, 3D chess that's going on here on these decision 701 00:33:43,760 --> 00:33:46,840 logics. So I know we always just like 702 00:33:46,840 --> 00:33:49,560 look at these in AB decisions of oh, read in terms of write in 703 00:33:49,560 --> 00:33:52,400 terms of, but this is stuff that's kind of coming down from 704 00:33:52,400 --> 00:33:53,960 other angles. And if you're doing fewer 705 00:33:53,960 --> 00:33:56,680 drives, OK, there are bigger. What's the endurance than that? 706 00:33:56,680 --> 00:33:59,000 OK, it's scaling. These are the things you need to 707 00:33:59,000 --> 00:34:01,160 be thinking about. Now you've mentioned it a few 708 00:34:01,160 --> 00:34:03,960 times already, so I probably should ask the question. 709 00:34:03,960 --> 00:34:06,280 When it comes to memory tearing, is it something that comes up 710 00:34:06,520 --> 00:34:10,320 with customers fairly regularly? Are people actually adopting it? 711 00:34:10,320 --> 00:34:12,719 And of course, what is the reason for it? 712 00:34:12,719 --> 00:34:18,560 But I guess we already know. So yes, so one of the largest 713 00:34:18,560 --> 00:34:21,360 customers that I've talked to recently, you know, if we go 714 00:34:21,360 --> 00:34:24,880 back 18 months ago, people were buying RAM, small, small dams 715 00:34:24,880 --> 00:34:29,840 for like $5 a gig, maybe $10 a gig on the denser stuff. 1 716 00:34:29,840 --> 00:34:32,080 customer I talked to recently and these guys have, I'll say, 717 00:34:32,080 --> 00:34:34,679 some purchasing power. Like we're talking like, you 718 00:34:34,679 --> 00:34:36,760 know, 910 figure orders here of RAM. 719 00:34:37,239 --> 00:34:39,960 I think they were paying like $40 a GB. 720 00:34:41,080 --> 00:34:44,760 And when we first came out with the feature memory tie ring 721 00:34:44,760 --> 00:34:48,360 memory was like half the cost of a host or something. 722 00:34:48,360 --> 00:34:52,040 When you get to these RAM dense builds, memory is like 90% of 723 00:34:52,040 --> 00:34:54,679 the host cost. Like Intel could come out with 724 00:34:54,679 --> 00:34:58,000 another CPU that's like, hey, this is an extra 20% clock 725 00:34:58,000 --> 00:34:58,840 speed. We're going to charge you an 726 00:34:58,840 --> 00:35:00,760 extra 3 grand. And that would just be a 727 00:35:00,760 --> 00:35:02,800 rounding error on some of these server builds. 728 00:35:02,800 --> 00:35:04,800 It's, it's, it's crazy where we're at. 729 00:35:05,640 --> 00:35:09,960 And so memory is basically driving everything and it's, 730 00:35:09,960 --> 00:35:12,080 it's not getting, you know, people say, well, it's just 731 00:35:12,080 --> 00:35:14,120 going to get better tomorrow. Well, there's only three memory 732 00:35:14,120 --> 00:35:16,000 vendors left. I mean, you got SK Hynix, you 733 00:35:16,000 --> 00:35:18,920 got Samsung, you got Micron. There used to be 17. 734 00:35:18,920 --> 00:35:23,160 And as you know, one of the Intel CEO's said in the 80s when 735 00:35:23,160 --> 00:35:25,280 he was leaving memory like only the paranoid survive. 736 00:35:25,280 --> 00:35:28,000 And it's, it's very true like these guys, it takes, it's 737 00:35:28,040 --> 00:35:31,600 multiple years to spin up, you know, their ASML orders to get 738 00:35:31,600 --> 00:35:33,160 fabs, to get the downstream supplies. 739 00:35:33,440 --> 00:35:35,840 This is something that takes, you know, they, they have been 740 00:35:35,840 --> 00:35:38,400 through many boom bust cycles and not everybody wants 741 00:35:38,600 --> 00:35:41,040 everybody looks at the other, you know, 14 vendors that aren't 742 00:35:41,040 --> 00:35:44,560 there and has a memory. So this isn't going away 743 00:35:44,560 --> 00:35:46,360 quickly. This is RAM is getting very 744 00:35:46,360 --> 00:35:48,040 expensive. And so memory tie ring is 745 00:35:48,040 --> 00:35:50,400 basically in every customer conversation I have. 746 00:35:50,480 --> 00:35:53,080 If you, you know, if you were, if you're a partner or you're an 747 00:35:53,080 --> 00:35:55,640 SE or someone, it should be in every customer conversation you 748 00:35:55,640 --> 00:35:56,640 have. If you're a customer, you should 749 00:35:56,640 --> 00:36:00,240 be asking about it because if you can cut that RAM bill in 750 00:36:00,240 --> 00:36:03,480 half on that dense order from 40,000 to 20,000 or even from 751 00:36:03,480 --> 00:36:08,840 20,000 to 10,000, that's gonna that's gonna help offset some of 752 00:36:08,840 --> 00:36:10,800 that sting. And you may say, well, isn't 753 00:36:10,800 --> 00:36:13,520 NVMU a lot slower than ramp? Yeah, it is. 754 00:36:13,520 --> 00:36:17,480 But we have a really good hypervisor at identifying which 755 00:36:17,480 --> 00:36:19,440 pages are active and which pages are idle. 756 00:36:20,360 --> 00:36:23,640 And I got a hold of a phone home data set with a couple million 757 00:36:23,640 --> 00:36:26,520 workloads and actually saw an aggregate average memory page 758 00:36:26,520 --> 00:36:28,760 activity. And the median environment, it's 759 00:36:28,760 --> 00:36:32,120 about 20%. So even tiearing 1 to 150%, 760 00:36:32,440 --> 00:36:35,080 that's actually conservative. Like let's start there, please. 761 00:36:35,080 --> 00:36:37,040 Let's start there and let's start with your tier, you know, 762 00:36:37,040 --> 00:36:38,200 1-2 apps. Let's start with your 763 00:36:38,200 --> 00:36:40,760 applications, your VDI, it will work our way up. 764 00:36:40,760 --> 00:36:44,320 But I've talked to people, Brandon Frost spoke at VM World 765 00:36:44,320 --> 00:36:46,760 and he was using the tech preview and he was running 1 to 766 00:36:46,760 --> 00:36:50,600 one with with sequel server and he's like, yeah, at a 10% 767 00:36:50,600 --> 00:36:52,920 performance head. And I cut my RAM bill in half. 768 00:36:52,920 --> 00:36:56,600 Like that's, that's absurdly compelling. 769 00:36:56,800 --> 00:36:59,400 You know, that is that is selling free money, so to speak. 770 00:36:59,560 --> 00:37:01,560 And we, you know, how do we do it? 771 00:37:01,560 --> 00:37:03,760 We have all this IP around V motion and DRS. 772 00:37:03,760 --> 00:37:06,320 We've been tracking memory page activity better than anybody for 773 00:37:06,320 --> 00:37:09,000 longer. This is why your V motions don't 774 00:37:09,000 --> 00:37:11,800 stun, you know, and they and they they also don't stun a lot 775 00:37:11,800 --> 00:37:13,960 less when you buy better Knicks going full circle at the 776 00:37:13,960 --> 00:37:15,320 beginning of this podcast. So. 777 00:37:15,840 --> 00:37:19,040 Yeah, I actually spoke with Brendan at the event and he had 778 00:37:19,040 --> 00:37:21,040 a fantastic use case and a great story. 779 00:37:21,040 --> 00:37:23,800 So I was very happy for him to be up on stage and explaining 780 00:37:24,240 --> 00:37:25,720 how how we actually use the product. 781 00:37:25,720 --> 00:37:29,560 And I'll make sure to to add a link to that particular session 782 00:37:29,560 --> 00:37:32,400 in the in the show notes. Now, before I actually let let 783 00:37:32,400 --> 00:37:35,640 you go, one of the things I also briefly wanted to go over is for 784 00:37:35,640 --> 00:37:39,440 people that actually can still procure new hardware, how do 785 00:37:39,440 --> 00:37:41,080 they actually go about doing that? 786 00:37:41,080 --> 00:37:43,760 I mean, we know there are different ready note profiles, 787 00:37:44,040 --> 00:37:46,640 but what is your take on that? I I always have a very simple 788 00:37:46,640 --> 00:37:49,880 approach, but some customers take a very complicated approach 789 00:37:50,200 --> 00:37:53,960 where they look very specific to which ready note they need and 790 00:37:53,960 --> 00:37:56,120 then they get confused about what they can change and what 791 00:37:56,120 --> 00:37:58,960 they cannot change. So how do you go about that 792 00:37:58,960 --> 00:38:00,720 whole process when you talk to customers? 793 00:38:00,720 --> 00:38:04,680 There's kind of two ways to buy there is the like, I'm going to 794 00:38:04,680 --> 00:38:06,920 do this the nerdiest way possible, which I'm inherently 795 00:38:06,920 --> 00:38:09,480 drawn to because I, I love staring at these stats, which is 796 00:38:09,480 --> 00:38:11,080 like, OK, what are your resources? 797 00:38:11,840 --> 00:38:13,880 What are your constraints? How many nodes do we have? 798 00:38:14,200 --> 00:38:16,280 OK, how many racks, how many ports do I have? 799 00:38:16,280 --> 00:38:18,720 Like sometimes people work backwards on cluster size from 800 00:38:18,720 --> 00:38:20,440 top of rack ports and subscription ratios. 801 00:38:20,440 --> 00:38:22,440 This all just works. This is why you have to learn a 802 00:38:22,440 --> 00:38:23,960 little bit of everything, unfortunately. 803 00:38:24,760 --> 00:38:27,760 And then I go, OK, how do I, you know, fit those puzzle pieces 804 00:38:27,760 --> 00:38:29,880 together with this many megahertz, this many gigabytes 805 00:38:29,880 --> 00:38:32,720 of RAM, this much capacity? What's my constraint, you know, 806 00:38:32,720 --> 00:38:35,880 in the VCN sizer and which is evolving in the VCF sizer? 807 00:38:35,880 --> 00:38:38,000 By the way, I was on design calls for that today. 808 00:38:39,600 --> 00:38:42,280 And, and these are the things to where it's like, OK, how do I 809 00:38:42,280 --> 00:38:43,920 pack this in the most efficient way possible? 810 00:38:43,920 --> 00:38:47,680 Because I don't want to be in a situation where I decided that I 811 00:38:47,680 --> 00:38:50,000 was always going to buy nodes with two terabytes of RAM and 812 00:38:50,000 --> 00:38:52,840 then that causes me to double the size of my order, which now 813 00:38:52,840 --> 00:38:55,040 is going to cost me 3 quadrillion dollars. 814 00:38:55,440 --> 00:38:57,840 The other way people buy which people when when I feel like 815 00:38:57,840 --> 00:39:01,000 hardware was cheaper, people had kind of fallen into was OK, 816 00:39:01,000 --> 00:39:02,520 we're going to have our procurement department, we're 817 00:39:02,520 --> 00:39:04,400 going to figure out one or two nodes we're just going to run 818 00:39:04,400 --> 00:39:06,960 our entire fleet on. We're going to pick one or two 819 00:39:07,400 --> 00:39:11,000 Lego bricks, you know, and yeah, we're just going to order like a 820 00:39:11,000 --> 00:39:12,560 bazillion of those. And we'll just take our 821 00:39:12,560 --> 00:39:15,880 workloads and like, you know, play Tetris into those bricks. 822 00:39:17,840 --> 00:39:20,960 And that latter 1 used to be more popular, but I, I, you 823 00:39:20,960 --> 00:39:24,240 know, one, don't overthink and like, try to have a bespoke for 824 00:39:24,240 --> 00:39:26,520 every single node or node in a cluster or cluster. 825 00:39:26,760 --> 00:39:29,880 Like do try to find some kind of ratios, but then also pick 826 00:39:29,880 --> 00:39:32,400 outlier workloads and build separate different clusters to 827 00:39:32,400 --> 00:39:34,360 them. But I, I caution people in that 828 00:39:34,360 --> 00:39:38,000 ladder to where people are like, well, we've always bought ADL 829 00:39:38,000 --> 00:39:41,160 whatever with this number. And we do this. 830 00:39:41,800 --> 00:39:44,000 And often times when I ask people why they're like, well, 831 00:39:44,000 --> 00:39:47,760 our procurement apartment goes to, you know, they do an RFP 832 00:39:47,760 --> 00:39:51,560 once every six years and then we get a, a discount percentage and 833 00:39:51,560 --> 00:39:55,600 then we just do that. And I'm like, you're probably 834 00:39:55,640 --> 00:39:58,320 potentially wasting a lot of money having this situation 835 00:39:58,320 --> 00:40:00,160 where you're just trying to shove everything into that. 836 00:40:00,160 --> 00:40:03,040 And I encourage you if you are doing that, look at OPS. 837 00:40:04,480 --> 00:40:07,400 Look at, you know, how much CPU or memory, look at that ratio 838 00:40:07,400 --> 00:40:09,600 because they used to be something hardware's cheap. 839 00:40:09,600 --> 00:40:11,760 I don't care that hardware's getting really expensive, 840 00:40:11,760 --> 00:40:14,800 particularly in memory. And I would often look at memory 841 00:40:14,800 --> 00:40:17,640 and how that is bottlenecking me because the other stuff you can 842 00:40:17,640 --> 00:40:19,440 kind of adjust. You can adjust drive sizes, you 843 00:40:19,440 --> 00:40:22,160 can adjust cores per node, you can adjust down sockets. 844 00:40:22,160 --> 00:40:25,000 Actually single socket servers, by the way, we used to all like 845 00:40:25,200 --> 00:40:27,440 snuff our nose and be like, oh, those are for poor people or 846 00:40:27,440 --> 00:40:28,680 something. Why would I have a single 847 00:40:28,680 --> 00:40:30,600 socket? Single socket servers are 848 00:40:30,600 --> 00:40:31,920 actually awesome. Now you can actually get all 849 00:40:31,920 --> 00:40:34,320 memory channels, like all the weird things people crippled 850 00:40:34,520 --> 00:40:36,960 sockets on. So I'd encourage you to to 851 00:40:36,960 --> 00:40:40,440 revisit some of your priors. Look at this stuff, re evaluate. 852 00:40:40,680 --> 00:40:42,760 Don't just cargo cult and say we're going to buy the same 853 00:40:42,760 --> 00:40:44,320 thing because it always worked before. 854 00:40:45,880 --> 00:40:48,200 And look at this and, and, and try to tune this stuff and 855 00:40:48,200 --> 00:40:50,160 figure out what your primary cost is. 856 00:40:50,160 --> 00:40:52,240 Is that memory is it could be cores. 857 00:40:52,240 --> 00:40:54,400 Maybe you're buying an article rack and it costs 40,000 a core, 858 00:40:54,400 --> 00:40:56,760 but figure out what those business inputs are and work to 859 00:40:56,760 --> 00:40:58,520 that logic. So I know there's a long 860 00:40:58,520 --> 00:41:01,160 rambling answer, but there's a lot of inputs and you kind of 861 00:41:01,160 --> 00:41:03,240 need to learn them all and and put them together. 862 00:41:03,760 --> 00:41:07,440 Before I let you go, any famous last words or any final thoughts 863 00:41:07,440 --> 00:41:08,680 you would like to share with the audience? 864 00:41:10,000 --> 00:41:13,040 Please put TPMS in your servers before you install ESXI. 865 00:41:13,040 --> 00:41:15,320 It's, it's, it's going to make your life easier in the future. 866 00:41:15,360 --> 00:41:20,000 But, but, but also just, you know, the I, I wasn't always a 867 00:41:20,000 --> 00:41:21,920 networking guy. You know how I learned about 868 00:41:21,920 --> 00:41:23,560 networking? I went to lunch with those guys. 869 00:41:23,560 --> 00:41:25,840 Go to lunch with your peers. Go if you're not a storage guy, 870 00:41:25,840 --> 00:41:27,600 go hang out. The storage guys learn, learn 871 00:41:27,600 --> 00:41:29,440 what an IOP is, learn what your ratios are. 872 00:41:29,680 --> 00:41:32,480 Be curious, it'll help you in your career. 873 00:41:32,480 --> 00:41:34,600 It'll help you be a better view, more admin. 874 00:41:34,880 --> 00:41:36,760 And if you're trying to get approvals for budget right now, 875 00:41:36,760 --> 00:41:40,640 which is really hard, it's hard for me and everyone you know, 876 00:41:40,640 --> 00:41:43,040 I'm ordering new lab hardware for my house and got to get that 877 00:41:43,040 --> 00:41:45,440 through the the local budget process. 878 00:41:47,200 --> 00:41:49,360 By the way, you can buy used opt in cards for cheap from memory 879 00:41:49,360 --> 00:41:51,920 tearing on eBay. But these are the things like be 880 00:41:51,920 --> 00:41:54,280 curious, learn about these other domains adjacent to you and the 881 00:41:54,280 --> 00:41:57,760 resources and learn about how you're, you know, the the trades 882 00:41:57,760 --> 00:42:00,680 and the arbitrages and things between them to learn about your 883 00:42:00,680 --> 00:42:02,440 applications, figure out what's driving those workloads. 884 00:42:02,440 --> 00:42:05,200 It's going to make you a better architect and it's going to make 885 00:42:05,200 --> 00:42:07,000 it's going to make you learn some things. 886 00:42:07,000 --> 00:42:08,360 It'll be fun. And that's it. 887 00:42:08,520 --> 00:42:10,880 Thanks for tuning in to the Unexplored Territory podcast. 888 00:42:11,080 --> 00:42:13,440 If you enjoyed this episode, don't forget to subscribe and 889 00:42:13,440 --> 00:42:15,040 leave a review rating wherever possible. 890 00:42:15,040 --> 00:42:17,720 And please join us again next time as we cover more insights 891 00:42:17,720 --> 00:42:19,800 into cutting edge solutions shaping the world of IT. 892 00:42:20,280 --> 00:42:22,720 Until then, staying comfortable, keep exploring.