1 00:00:00,280 --> 00:00:02,400 Hi, and welcome to the Neil Ashton Podcast. 2 00:00:03,080 --> 00:00:05,920 In each episode, we explained some of the fascinating ways 3 00:00:05,920 --> 00:00:09,080 that science and engineering are changing the world around us. 4 00:00:09,800 --> 00:00:12,760 We talked to leading engineers from elite level sports like 5 00:00:12,840 --> 00:00:16,880 cycling and Formula One to some of the world's top academics to 6 00:00:16,880 --> 00:00:20,450 understand how fluid dynamics, machine learning and 7 00:00:20,450 --> 00:00:23,000 supercomputing are bringing in a new era of discovery. 8 00:00:23,960 --> 00:00:27,080 We also hear some of their life stories, their career advice, 9 00:00:27,640 --> 00:00:30,080 and lessons they've learned on the way that I hope will be 10 00:00:30,080 --> 00:00:33,800 helpful to you too. So sit back and enjoy this 11 00:00:33,800 --> 00:00:41,040 episode. Hi, and welcome back to the Neil 12 00:00:41,040 --> 00:00:43,760 Ashton Podcast. So today I wanted to talk about 13 00:00:43,760 --> 00:00:48,008 the topic of foundational models for fluids or for 14 00:00:48,008 --> 00:00:51,400 computational fluid dynamics. It's a topic that I brought up 15 00:00:51,960 --> 00:00:53,840 and asked quite a few of the people that I interviewed 16 00:00:53,840 --> 00:00:56,440 recently. But I, I wanted to give some 17 00:00:56,440 --> 00:01:00,760 personal thoughts on this, partially because I had the 18 00:01:00,760 --> 00:01:05,080 pleasure a couple of weeks ago of chairing a panel discussion 19 00:01:05,640 --> 00:01:10,680 at the second data-driven fluid dynamics conference, which was 20 00:01:10,680 --> 00:01:13,884 an ERCOFTAC Euromech thing that was done at Imperial 21 00:01:13,884 --> 00:01:16,920 College London. I was involved in the first one 22 00:01:16,920 --> 00:01:20,000 in Paris the previous year. And it's a great initiative that 23 00:01:20,000 --> 00:01:24,080 has tried to bring the fluids community and I guess the AI 24 00:01:24,080 --> 00:01:27,565 community together, particularly with a sort of European 25 00:01:27,565 --> 00:01:31,440 flavour, I guess to it. But in this panel session, it 26 00:01:31,440 --> 00:01:35,160 was really interesting because we had representatives from, you 27 00:01:35,160 --> 00:01:41,233 know, Harvard, but also Caltech and, and some from industry 28 00:01:41,233 --> 00:01:46,400 and, and really representing the experimental and the 29 00:01:46,400 --> 00:01:52,280 computational side. And one of the things that I did 30 00:01:52,280 --> 00:01:56,200 was to try and ask people about their opinion of foundational 31 00:01:56,200 --> 00:02:02,240 models. But more interestingly or as 32 00:02:02,240 --> 00:02:06,480 interesting was I tried to ask the audience some of the 33 00:02:06,480 --> 00:02:09,926 questions and there was a couple of hundred people in the 34 00:02:09,926 --> 00:02:11,572 audience. And so we use this Mentimeter 35 00:02:11,580 --> 00:02:14,320 app that I have to confess I haven't really used before, but 36 00:02:14,320 --> 00:02:17,148 it's great because you can essentially do real-time polling 37 00:02:17,148 --> 00:02:21,069 of people. So I asked the question, do you 38 00:02:21,069 --> 00:02:23,548 believe that we will have foundational models? 39 00:02:23,556 --> 00:02:27,536 And in fact, I'm just looking on the on the screen now because I 40 00:02:27,536 --> 00:02:29,598 have the results up. Well, I should tell you, the 41 00:02:29,606 --> 00:02:32,748 first thing I did ask was what's your favorite British food? 42 00:02:32,756 --> 00:02:37,392 Bit of a tongue-in-cheek really. And fish and chips won, 43 00:02:37,392 --> 00:02:41,596 followed secondly by the vomit emoji, which I was a bit 44 00:02:41,604 --> 00:02:44,040 offended by. And then a full English 45 00:02:44,048 --> 00:02:46,748 breakfast. So there we go. 46 00:02:46,756 --> 00:02:51,440 But more seriously, on the actual topic of foundational 47 00:02:51,448 --> 00:02:57,268 models, I basically said, will you know, can we get 48 00:02:57,276 --> 00:02:59,779 foundational model for fluids? How generalisable could it be? 49 00:02:59,787 --> 00:03:04,756 And what I asked was, could it? Could this foundational model 50 00:03:04,764 --> 00:03:08,389 provide instantaneous pressure, velocity for any boundary 51 00:03:08,397 --> 00:03:12,254 condition or geometry? So as in you can say to it 52 00:03:12,262 --> 00:03:15,288 here's a geometry with a certain boundary condition, can it 53 00:03:15,296 --> 00:03:18,480 predict it and give you the instantaneous pressure and 54 00:03:18,488 --> 00:03:20,097 velocity? A bit like a CFD solver. 55 00:03:20,105 --> 00:03:22,200 Now you give it a geometry, you give it some boundary 56 00:03:22,208 --> 00:03:24,220 conditions, and it would go and solve it. 57 00:03:24,228 --> 00:03:29,500 Second option I gave people is exactly the same, but time 58 00:03:29,508 --> 00:03:31,355 averaged. So not instantaneous, but time 59 00:03:31,363 --> 00:03:34,884 averaged, and I'll get into the nuance of that in a little bit. 60 00:03:34,884 --> 00:03:36,600 Then the third option was yes, 61 00:03:36,600 --> 00:03:40,240 but only for a specific use case, as in maybe like a road, 62 00:03:40,240 --> 00:03:43,400 car or a plane. And finally, option was 63 00:03:44,160 --> 00:03:47,120 essentially no physics. Sorry fluids are just too 64 00:03:47,120 --> 00:03:48,760 complex for an AI model to learn. 65 00:03:49,280 --> 00:03:55,240 And interestingly, out of the scores it was 38 and this is out 66 00:03:55,240 --> 00:03:58,480 of 100 and something. So the the the most popular 67 00:03:58,480 --> 00:04:04,280 answer was only for a specific use case. 38 people close 2nd. 68 00:04:04,280 --> 00:04:08,535 37 was provide time average to any boundary condition or 69 00:04:08,535 --> 00:04:12,040 geometry followed by instantaneous pressure. 70 00:04:12,080 --> 00:04:15,120 And finally 13 had fluids that are just too complex. 71 00:04:15,360 --> 00:04:17,959 So out of that audience, which is biased because they are 72 00:04:18,440 --> 00:04:24,480 researchers looking at fluids and AI, the sense was it really 73 00:04:24,480 --> 00:04:26,760 is only possible for a specific use case. 74 00:04:26,760 --> 00:04:30,920 And therefore the terminology of foundational model is perhaps 75 00:04:31,440 --> 00:04:34,320 pushing it a little bit or needs redefining. 76 00:04:35,720 --> 00:04:37,360 OK. So that's why essentially I 77 00:04:37,360 --> 00:04:43,400 wanted to to talk about it and I wanted to share I guess some 78 00:04:43,400 --> 00:04:47,120 opinions of my own, some stuff I've seen along the way. 79 00:04:47,480 --> 00:04:48,920 I'm in a lucky position, I guess. 80 00:04:49,520 --> 00:04:52,200 I'm working in the tech sector that I get to engage with a lot 81 00:04:52,200 --> 00:04:54,680 of customers and academics from different groups. 82 00:04:54,680 --> 00:04:58,400 So I like to go to conferences. 83 00:04:58,400 --> 00:05:00,889 And I'd like to say I'd therefore have a reasonably good 84 00:05:00,889 --> 00:05:02,880 view of what's going on in the community. 85 00:05:03,560 --> 00:05:07,040 And I would actually say that things are looking very 86 00:05:07,040 --> 00:05:12,764 interesting as in only a couple of years ago there was really 87 00:05:12,764 --> 00:05:15,960 no work into this idea of a foundational model. 88 00:05:17,160 --> 00:05:19,880 And another way of thinking of foundational model is simply a 89 00:05:19,880 --> 00:05:23,040 pretrained model. So what most people were looking 90 00:05:23,040 --> 00:05:26,240 at was simply can we come up with some sort of algorithm, 91 00:05:26,240 --> 00:05:29,840 some architecture that could allow somebody to go and train 92 00:05:29,840 --> 00:05:32,200 an AI model based on prior simulation data. 93 00:05:33,240 --> 00:05:35,680 So we're not talking about accelerating an existing code, 94 00:05:35,680 --> 00:05:37,920 we're talking about replacing the code with a surrogate model. 95 00:05:38,760 --> 00:05:41,804 And that was certainly the main focus probably five years ago 96 00:05:41,804 --> 00:05:44,920 or four years ago. And there was some seminal work 97 00:05:44,920 --> 00:05:47,757 done with things like Mesh- GraphNets and others which took 98 00:05:47,757 --> 00:05:50,472 a graph neural net approach where you could essentially 99 00:05:50,472 --> 00:05:53,390 take the nodes of the mesh as the input. 100 00:05:53,398 --> 00:05:56,744 And that was something or particles I guess because it was 101 00:05:56,744 --> 00:06:00,435 all particle examples of this. And this rapidly became one of 102 00:06:00,443 --> 00:06:03,440 the success stories in the weather and climate, but also 103 00:06:03,448 --> 00:06:08,270 was showing potential for CFD. And this led to a number of 104 00:06:08,278 --> 00:06:12,190 start-ups that that formed over the time, the sort of NAVASTO, 105 00:06:12,198 --> 00:06:16,120 Neural Concept and others who took some of this work or 106 00:06:16,128 --> 00:06:19,550 pioneered things themselves and, and released some of those 107 00:06:19,558 --> 00:06:22,560 things that people could go and, and try for themselves. 108 00:06:22,568 --> 00:06:26,672 And so you saw in the past, you know, several years, people 109 00:06:26,680 --> 00:06:33,315 really wanting to test out those methods and with some success 110 00:06:33,323 --> 00:06:39,045 for sure. But obviously that requires you 111 00:06:39,045 --> 00:06:44,895 to have your own training data, your own ability to train a 112 00:06:44,903 --> 00:06:48,765 model and to have, I guess, both the software and the knowledge 113 00:06:48,773 --> 00:06:54,009 to do it. And I guess if you can compare 114 00:06:54,017 --> 00:06:57,476 that against what you would typically see in the LLM space 115 00:06:57,484 --> 00:07:02,180 where the majority of us are not training our own LLMs, but we 116 00:07:02,188 --> 00:07:06,125 are simply doing a prompt and asking a question, doing an 117 00:07:06,133 --> 00:07:08,020 inference, we're not training it. 118 00:07:08,028 --> 00:07:12,836 The logical question was, well, could it be possible to train a 119 00:07:12,836 --> 00:07:16,627 model and then just provide it to people as a pretrained 120 00:07:16,635 --> 00:07:19,960 model? And it's a really interesting 121 00:07:19,968 --> 00:07:25,355 discussion because it has pretty large it, it could have a 122 00:07:25,363 --> 00:07:29,424 potentially very large impact on the CFD community. 123 00:07:29,432 --> 00:07:33,755 Now, there are some of you out there who still have very 124 00:07:33,763 --> 00:07:36,748 negative opinions of AI and sort of seeing it as a fad, and it 125 00:07:36,748 --> 00:07:40,300 will come and go. But I would say those voices are 126 00:07:40,300 --> 00:07:43,576 slowly starting to be overtaken by the realization 127 00:07:43,576 --> 00:07:47,115 that actually these methods have a lot of potential. 128 00:07:47,123 --> 00:07:52,467 Only, like I said, five years ago this was basically not 129 00:07:52,475 --> 00:07:54,354 happening. Nobody was even talking about 130 00:07:54,362 --> 00:07:57,622 it. I remember I got involved maybe 131 00:07:57,622 --> 00:08:01,418 three years ago in this topic, so still quite late to the game 132 00:08:01,418 --> 00:08:04,702 I guess. But even three years ago I 133 00:08:04,710 --> 00:08:08,875 remember most people had no idea what this was all about. 134 00:08:08,883 --> 00:08:12,357 They just thought of this as like reduced order modelling 135 00:08:12,365 --> 00:08:17,035 ROMs, really no sense of what was going on, didn't know what 136 00:08:17,043 --> 00:08:20,490 architectures would work. There was only maybe one or two 137 00:08:20,490 --> 00:08:22,574 start-ups in the space, wasn't very prominent. 138 00:08:22,582 --> 00:08:29,005 Fast forward to now and I would say that it is certainly rapidly 139 00:08:29,005 --> 00:08:33,294 expanding and I would say companies can be categorised as 140 00:08:33,294 --> 00:08:37,826 either trying to lead in this way, being very interested and 141 00:08:37,833 --> 00:08:42,830 the number who were just set against it is now sort of 142 00:08:42,837 --> 00:08:47,657 getting lower. But I would say that this is 143 00:08:47,665 --> 00:08:50,336 still a little bit split between the industries. 144 00:08:50,344 --> 00:08:54,455 And like with there's a bit of a parallel with high fidelity 145 00:08:54,463 --> 00:08:58,990 methods, I see the automotive sector as being much faster to 146 00:08:58,998 --> 00:09:02,150 adopt these new technologies or investigate these new 147 00:09:02,158 --> 00:09:05,189 technologies. Automotive companies today use 148 00:09:05,197 --> 00:09:09,290 hybrid RANS, scale- resolving methods on a day-to-day basis, 149 00:09:09,290 --> 00:09:13,230 whereas the aerospace sector is still sort of proving out that 150 00:09:13,230 --> 00:09:15,600 they think they can be used in in production. 151 00:09:15,800 --> 00:09:17,920 And some of that is because Reynolds numbers and 152 00:09:17,920 --> 00:09:20,760 complexities. Another is just a mindset, I 153 00:09:20,760 --> 00:09:22,840 believe. And interestingly, I see the 154 00:09:22,840 --> 00:09:25,612 same thing with AI. I see that the automotive 155 00:09:25,612 --> 00:09:29,454 companies are far more motivated to try and find AI approaches 156 00:09:29,454 --> 00:09:33,880 that can be faster potentially and are embracing it. 157 00:09:33,880 --> 00:09:37,280 Whereas I see aerospace is more in the mindset of I don't think 158 00:09:37,280 --> 00:09:40,360 this can work for me. And I guess part of me, the 159 00:09:40,360 --> 00:09:43,320 reason doing this podcast episode is really just to put 160 00:09:43,320 --> 00:09:46,120 across that, and this is a genuine, this is not influenced 161 00:09:46,120 --> 00:09:49,840 by the company I work for. These are all sort of 162 00:09:49,840 --> 00:09:53,880 independent opinions, personal thoughts, but I really do think 163 00:09:53,880 --> 00:09:57,600 that these are looking incredibly interesting for the 164 00:09:57,600 --> 00:10:01,040 obvious reason that, you know, now if you do an inference on a, 165 00:10:01,480 --> 00:10:04,560 on a model, you're going to get an answer in seconds. 166 00:10:06,600 --> 00:10:12,160 But the accuracy is debatable whether people have truly proved 167 00:10:12,160 --> 00:10:15,200 out that these these AI methods are accurate enough. 168 00:10:15,440 --> 00:10:18,495 They give a very good impression of, you know, approximately 169 00:10:18,495 --> 00:10:20,720 what the flow field is or approximately what the lift or 170 00:10:20,720 --> 00:10:23,080 drag is. But I would say that we are 171 00:10:23,080 --> 00:10:26,440 still not at the case where they're widely accepted or there 172 00:10:26,440 --> 00:10:30,960 are very clear accuracy. I would say it's a bit like CFD 173 00:10:31,000 --> 00:10:34,160 in the early days where you could argue there was clear 174 00:10:34,320 --> 00:10:37,120 points where it worked well, but there were many examples where 175 00:10:37,120 --> 00:10:40,760 it didn't work. And hence why CFD took decades, 176 00:10:40,920 --> 00:10:45,080 you know, to to progress and be trusted enough to be a dominant 177 00:10:45,080 --> 00:10:48,386 design tool. I'd say AI is still in that 178 00:10:48,386 --> 00:10:52,400 sense where it's probably not trusted as a key design tool 179 00:10:52,400 --> 00:10:55,499 yet. But I would say that the number 180 00:10:55,499 --> 00:10:58,140 of companies really investing in this is significantly 181 00:10:58,140 --> 00:11:01,080 growing. The number of start-ups who are 182 00:11:01,080 --> 00:11:03,800 emerging in this space is growing month on month. 183 00:11:04,560 --> 00:11:08,280 And the interest from the big ISVs is also growing month on 184 00:11:08,280 --> 00:11:15,560 month. But the challenge is the data. 185 00:11:16,080 --> 00:11:19,560 So do you have available data? And this has always been the 186 00:11:19,560 --> 00:11:23,960 challenge for the idea of a foundational model, as probably 187 00:11:23,960 --> 00:11:26,833 people know, you know, the reason that these LLMs and 188 00:11:26,833 --> 00:11:30,289 others can do so well is that they have a very large data 189 00:11:30,289 --> 00:11:34,680 source to, to to, to get to. So typically scraping off the 190 00:11:34,680 --> 00:11:39,360 Internet, but nowadays maybe doing commercial deals with a 191 00:11:39,360 --> 00:11:42,000 certain repository of data. You know, if you want to do 192 00:11:42,000 --> 00:11:44,787 something on journal papers, you might do a deal with Elsevier 193 00:11:44,787 --> 00:11:47,320 or John Wiley. If you want to do something on 194 00:11:47,320 --> 00:11:50,840 video, you might do a deal with a video, you know, distributor 195 00:11:51,680 --> 00:11:54,240 or or a newspaper. If you want to do on text or 196 00:11:54,240 --> 00:11:55,800 pictures, you might do with Adobe. 197 00:11:56,280 --> 00:11:58,600 You know, there's a lot of commercial arrangements going on 198 00:11:58,600 --> 00:12:00,600 now. And I think that's what's super 199 00:12:00,600 --> 00:12:04,600 interesting on the CFD side because on one side you could 200 00:12:04,600 --> 00:12:07,440 say the data is just not available. 201 00:12:07,440 --> 00:12:10,600 If you want to do combustion modelling or multiphase or 202 00:12:10,600 --> 00:12:13,920 hypersonic something. This is not publicly available 203 00:12:13,920 --> 00:12:18,120 data. So how do you you know, how do 204 00:12:18,120 --> 00:12:20,960 you solve that? And this is where I think 205 00:12:20,960 --> 00:12:25,360 there's an interesting technical but also commercial angle to 206 00:12:25,360 --> 00:12:31,800 this, which is, is it worth it for a company to independently 207 00:12:31,920 --> 00:12:36,868 run lots and lots of simulations to the point that they can 208 00:12:36,868 --> 00:12:41,240 pre- train a model and then supply the model and, and and 209 00:12:41,240 --> 00:12:46,560 charge it at a certain rate. And we have seen some early 210 00:12:46,560 --> 00:12:49,280 signs of this happening. You know, there's a start up 211 00:12:49,280 --> 00:12:51,960 Luminary Cloud. Some people know that I spoke to 212 00:12:52,120 --> 00:12:55,970 to Juan Alonso, the founder, and they recently released a sort 213 00:12:55,970 --> 00:12:59,332 of foundational model for road cars where they'd, you know, 214 00:12:59,332 --> 00:13:03,045 run a whole bunch of simulations themselves in collaboration 215 00:13:03,045 --> 00:13:08,859 with an end customer of theirs and and then released that and 216 00:13:08,859 --> 00:13:12,360 pretrained the model. Now that it's probably too early 217 00:13:12,360 --> 00:13:15,160 to see, you know, how ultimately successful that that that will 218 00:13:15,160 --> 00:13:17,732 be. But they've sort of fired the 219 00:13:17,732 --> 00:13:22,084 starting gun, so to speak, on an actual company releasing a 220 00:13:22,084 --> 00:13:25,120 sort of pretrained model. And I think it's really 221 00:13:25,120 --> 00:13:29,400 interesting to observe how useful that is because on one 222 00:13:29,400 --> 00:13:33,040 side, it removes a massive barrier for a company to to 223 00:13:33,040 --> 00:13:35,994 collect all the data, find all the data to have the expertise 224 00:13:35,994 --> 00:13:39,040 to use an AI tool. It's much easier just to do 225 00:13:39,080 --> 00:13:40,900 inference on a pretrained model. 226 00:13:40,908 --> 00:13:45,114 So I think that's the first thing I predict that we're going 227 00:13:45,114 --> 00:13:47,980 to see many more of those come out. 228 00:13:47,988 --> 00:13:53,280 Many more companies will will, will do that both I think from 229 00:13:53,280 --> 00:14:00,775 the startup and ISV space. But how, how much data can a 230 00:14:00,783 --> 00:14:04,276 company generate and how can they incentivize it? 231 00:14:04,284 --> 00:14:08,104 Well, one interesting thing, when I spoke with Prith, who's 232 00:14:08,112 --> 00:14:12,121 the CTO of Ansys, he said in one of the talks that I gave that, 233 00:14:12,129 --> 00:14:15,580 you know, maybe the ISV needs to incentivize people to allow the 234 00:14:15,580 --> 00:14:19,735 ISV to have the data. So I think that's a really 235 00:14:19,743 --> 00:14:22,464 interesting proposition that, you know, what if you would tick 236 00:14:22,464 --> 00:14:26,312 a box that was to say, well, you know, as you run your CFD 237 00:14:26,320 --> 00:14:30,960 simulation, I give permission for the ISV to use that data and 238 00:14:30,960 --> 00:14:33,704 perhaps in return they get, you know, some commercial 239 00:14:33,704 --> 00:14:37,570 incentive to do it. And I think that's a really 240 00:14:37,578 --> 00:14:39,728 interesting proposition. And I'm, I'm, I'm keen to see if 241 00:14:39,728 --> 00:14:41,835 some of the ISVs go down that route. 242 00:14:41,843 --> 00:14:47,932 And this actually links why I've often spoken about the cloud 243 00:14:47,940 --> 00:14:52,028 being an important avenue and the cloud being an important 244 00:14:52,028 --> 00:14:55,423 avenue links to this because of the SaaS bit. 245 00:14:55,431 --> 00:15:01,579 If you give somebody a on-prem binary, even if they tick some 246 00:15:01,587 --> 00:15:06,315 box to say we're happy for you to look at our data, what's the 247 00:15:06,315 --> 00:15:09,104 mechanism to share that file? Well, it's pretty difficult 248 00:15:09,112 --> 00:15:10,880 because it's running on their local network. 249 00:15:10,888 --> 00:15:13,735 How are they going to, you know, send that over to you? 250 00:15:13,743 --> 00:15:17,970 It's not practical where with a SaaS solution, it's much easier 251 00:15:17,970 --> 00:15:21,227 because it's by definition running in, let's say someone's 252 00:15:21,227 --> 00:15:23,824 cloud. And so if you say I want to 253 00:15:23,832 --> 00:15:27,006 share some of my data with them, it's actually much easier to do 254 00:15:27,006 --> 00:15:29,372 it. So I predict that part of the 255 00:15:29,380 --> 00:15:32,592 motivation and I think we'll see an acceleration of the 256 00:15:32,600 --> 00:15:38,010 SaaSification is to make this sort of AI and data collection 257 00:15:38,018 --> 00:15:40,702 easier both. If you're an existing AI start- 258 00:15:40,702 --> 00:15:44,541 up, you know, you could say, hey, use my AI tool and I'll 259 00:15:44,549 --> 00:15:48,546 give you a discount if you share your training data with me and 260 00:15:48,554 --> 00:15:50,590 over time will collect more data. 261 00:15:50,598 --> 00:15:57,085 And also it links into one of the things I've said repeatedly 262 00:15:57,085 --> 00:16:01,312 on this podcast about fast CFD solvers or CAE solvers. 263 00:16:01,320 --> 00:16:05,822 Because now one of the big things is if you've got to 264 00:16:05,830 --> 00:16:10,546 generate 5000 CFD cases, if your CFD solver is twice as fast as 265 00:16:10,554 --> 00:16:13,958 another CFD solver, that's a big amount of money to save. 266 00:16:13,966 --> 00:16:18,222 Or for the same budget, you could run twice as many cases. 267 00:16:18,230 --> 00:16:22,460 So if we assume that, to build these foundational models will 268 00:16:22,460 --> 00:16:26,160 have to run hundreds of thousands of cases, then 269 00:16:26,168 --> 00:16:31,540 actually there will be a huge focus on on enabling the code to 270 00:16:31,540 --> 00:16:36,057 be as efficient as possible. Now, one of the other things 271 00:16:36,065 --> 00:16:39,580 that I haven't brought up, but I think is another interesting one 272 00:16:39,580 --> 00:16:43,639 is everything I've spoken about now assumes that the CFD 273 00:16:43,639 --> 00:16:46,040 code is independent from the AI code. 274 00:16:46,640 --> 00:16:49,200 But as anybody knows anything about, for example, in-situ 275 00:16:49,200 --> 00:16:52,720 visualisation, we'll know that there's a strong move now 276 00:16:52,800 --> 00:16:56,040 towards this idea of doing it, you know, online rather than 277 00:16:56,040 --> 00:16:59,040 saving everything to disk and then doing the visualisation. 278 00:16:59,040 --> 00:17:00,480 Can you do it whilst it's running? 279 00:17:01,280 --> 00:17:04,400 And this is obviously something that's not just me, you know, 280 00:17:04,400 --> 00:17:07,040 saying this for the first time, it's known that I think 281 00:17:07,040 --> 00:17:09,619 there'll be a big increase, and there already is, certainly in 282 00:17:09,619 --> 00:17:13,359 the academic world of trying to build your CFD code in the 283 00:17:13,359 --> 00:17:15,040 same framework as your AI code. 284 00:17:15,048 --> 00:17:17,615 Some groups are doing this in PyTorch, you know, how do I 285 00:17:17,623 --> 00:17:20,492 write a code in there? Or they're using some sort of 286 00:17:20,500 --> 00:17:22,663 Python code that you can automatically, you know, 287 00:17:22,671 --> 00:17:26,773 differentiate and move between to pass gradients along and to 288 00:17:26,781 --> 00:17:30,016 the neural networks. But I predict this will be a 289 00:17:30,024 --> 00:17:33,900 will be important because if you are trying to build some 290 00:17:33,900 --> 00:17:37,277 pretrained model, some sort of foundational model, you don't 291 00:17:37,285 --> 00:17:41,352 really want to be doing your CFD independently to your AI 292 00:17:41,360 --> 00:17:43,992 training. You want them to be tightly 293 00:17:44,000 --> 00:17:48,259 coupled and potentially adding more points as you need them and 294 00:17:48,259 --> 00:17:52,881 not having to be, you know, bound by I/O issues, having to 295 00:17:52,889 --> 00:17:55,855 keep the data. Which is particularly true if 296 00:17:55,863 --> 00:17:59,705 you have the dream, and I think this is a much longer term dream 297 00:17:59,705 --> 00:18:01,760 of being able to do instantaneous. 298 00:18:01,768 --> 00:18:06,520 So time dependent, most training now is done on time average 299 00:18:06,528 --> 00:18:09,028 solutions, even if it's a time accurate simulation, because 300 00:18:09,036 --> 00:18:12,805 just the amount of data, I mean having created with colleagues, 301 00:18:12,805 --> 00:18:17,490 you know the DrivAerML dataset, it's already 30 terabytes with 302 00:18:17,498 --> 00:18:21,208 one time step essentially as in the time averaging, if you were 303 00:18:21,208 --> 00:18:24,520 going to try and do it for all of them, 200,000 iterations, 304 00:18:24,528 --> 00:18:28,588 well, you can do the maths, it's unbelievably large. 305 00:18:28,596 --> 00:18:33,688 But if at every time we were running those simulations, we 306 00:18:33,696 --> 00:18:38,118 were passing that data to a model to train, then, of course, 307 00:18:38,118 --> 00:18:40,910 we don't need to save the time steps out. 308 00:18:40,918 --> 00:18:43,700 We're just passing it in memory. So we actually could have done 309 00:18:43,708 --> 00:18:46,490 it because we were running the simulations time- dependent 310 00:18:46,490 --> 00:18:50,514 anyway. So I think this is where there's 311 00:18:50,514 --> 00:18:54,976 a lot of interest in these. I feel this is where new 312 00:18:54,984 --> 00:18:59,086 technologies are converging with AI as the sort of motivating 313 00:18:59,094 --> 00:19:01,374 factor. So foundational models. 314 00:19:01,382 --> 00:19:04,608 I, I really would encourage any academic who's listening to 315 00:19:04,616 --> 00:19:10,176 this, I think this is the topic to propose as a, you know, a big 316 00:19:10,176 --> 00:19:13,296 university project or European project or government project 317 00:19:13,304 --> 00:19:15,932 or, you know, anything that is big. 318 00:19:15,940 --> 00:19:20,258 Because if this could be made and there's so many debates on 319 00:19:20,258 --> 00:19:22,880 whether it should be open source, should be closed source. 320 00:19:22,880 --> 00:19:25,880 It could transform the way that 321 00:19:25,880 --> 00:19:29,720 we are doing CFD today. If it is possible, it would 322 00:19:29,720 --> 00:19:32,600 change the commercial landscape, it would change the technical 323 00:19:32,600 --> 00:19:35,400 landscape. And if you imagine that it's 324 00:19:35,400 --> 00:19:39,480 possible for a certain class of applications, you've got to 325 00:19:39,480 --> 00:19:41,640 imagine the transfer learning, the fine tuning. 326 00:19:41,760 --> 00:19:46,240 You know, at some point how different is a car than a plane 327 00:19:46,280 --> 00:19:50,240 or a city if you are starting with, if the model is able to 328 00:19:50,240 --> 00:19:51,720 learn some of these interactions. 329 00:19:52,240 --> 00:19:54,720 I haven't really spoken about the idea of including physics 330 00:19:54,720 --> 00:20:01,200 into it, but I I really do feel that just as five years ago AI 331 00:20:01,200 --> 00:20:04,120 was an interesting thing for people to look at, I think the 332 00:20:04,120 --> 00:20:08,280 whole concept of foundational models is becoming interesting. 333 00:20:08,320 --> 00:20:11,348 I believe at the beginning it will be for specific use cases 334 00:20:11,348 --> 00:20:18,600 as the audience voted, but I perceive that there will be a 335 00:20:18,600 --> 00:20:22,640 bit of an arms race between the ISVs and startups to build 336 00:20:22,640 --> 00:20:25,932 this, and it'll be interesting to see whether companies see 337 00:20:25,932 --> 00:20:29,864 this as their secret sauce. Whether you know, an aerospace 338 00:20:29,872 --> 00:20:32,545 manufacturer or an automotive manufacturer says, well, I don't 339 00:20:32,545 --> 00:20:35,155 need the ISV, I'm going to do it myself. 340 00:20:35,163 --> 00:20:39,556 And this will be an interesting balance between the software 341 00:20:39,564 --> 00:20:42,448 suppliers, the companies, because on the other hand, the 342 00:20:42,456 --> 00:20:44,660 automotive company can say, well, we're not a software, 343 00:20:44,668 --> 00:20:46,943 we're a car designer. Why are we going to write our 344 00:20:46,951 --> 00:20:49,200 own codes? And that's certainly been the 345 00:20:49,208 --> 00:20:51,814 case even in the aerospace. There's a move towards 346 00:20:51,822 --> 00:20:54,159 commercial codes. Well, you know, it used to be 347 00:20:54,167 --> 00:20:56,480 the case, everybody would write their own codes and that was 348 00:20:56,488 --> 00:20:58,800 their IP. And then they realised their IP 349 00:20:58,800 --> 00:21:01,268 is making cars or planes, not writing software. 350 00:21:01,276 --> 00:21:06,190 So it, I still suspect that most of this work, these foundation 351 00:21:06,190 --> 00:21:11,132 models will still make their way into commercial sort of big 352 00:21:11,132 --> 00:21:16,282 software companies, ISVs. But given that they don't have 353 00:21:16,290 --> 00:21:20,064 the data, it's the sort of engineering companies who have 354 00:21:20,072 --> 00:21:23,280 the data and they need to somehow get that data or produce 355 00:21:23,280 --> 00:21:24,720 that data. There'll be an interesting 356 00:21:24,728 --> 00:21:26,040 dynamic, I think in collaboration. 357 00:21:26,048 --> 00:21:28,943 So these are just my personal thoughts. 358 00:21:28,951 --> 00:21:32,739 I could be completely wrong, but I am quite bullish on the idea 359 00:21:32,747 --> 00:21:34,720 of some of these foundational models. 360 00:21:34,728 --> 00:21:39,308 And, and certainly, you know, in my day job, this is something 361 00:21:39,316 --> 00:21:42,495 I'm actively pursuing. I'm I'm academically interested. 362 00:21:42,495 --> 00:21:44,760 I'm interested in, you know, 363 00:21:44,760 --> 00:21:47,120 NVIDIA doing its bit to help things along the way. 364 00:21:48,320 --> 00:21:51,760 And yeah, I'll be really interested to see what happens 365 00:21:51,920 --> 00:21:54,240 in a few years time. So maybe I I'll try and set a 366 00:21:54,240 --> 00:21:57,360 reminder in two years to record another one and see how much of 367 00:21:57,360 --> 00:22:00,640 this. I was right on and maybe it was 368 00:22:00,760 --> 00:22:03,080 it'll all not happen and I was completely wrong. 369 00:22:03,080 --> 00:22:06,280 But I'm, I'm going to make a bet that in a couple of years we 370 00:22:06,280 --> 00:22:11,240 will have progressed quite a bit further than than than we have 371 00:22:11,480 --> 00:22:15,160 done to date. So with that, thanks for thanks 372 00:22:15,160 --> 00:22:16,760 for listening. I would really enjoy your 373 00:22:16,760 --> 00:22:18,000 comments. Let me know what you think if 374 00:22:18,000 --> 00:22:20,160 I'm completely wrong, if you have a different viewpoint on 375 00:22:20,160 --> 00:22:23,200 it, please put your comments in the YouTube or, or send me a 376 00:22:23,200 --> 00:22:25,400 message. And yeah, thanks for listening 377 00:22:25,400 --> 00:22:27,480 and hope you enjoyed it.