Hi, and welcome to the Neil Ashton Podcast.
In each episode, we explained some of the fascinating ways
that science and engineering are changing the world around us.
We talked to leading engineers from elite level sports like
cycling and Formula One to some of the world's top academics to
understand how fluid dynamics, machine learning, supercomputing
are bringing in a new era of discovery.
We also hear some of their life stories, their career advice,
the lessons they've learned on the way that I hope will be
helpful to you too. So sit back and enjoy this
episode. Hi, and welcome back to the Neil
Ashton Podcast. So today's guest is Professor
Ricardo Vinuesa. Ricardo is currently a Professor
of Aerospace Engineering at the University of Michigan, where
he leads research at the intersection of machine
learning, fluid mechanics, turbulence, and artificial
intelligence. Before joining Michigan, he was
a professor at KTH Royal Institute of Technology in
Stockholm, one of Europe's leading engineering
universities, and he built an internationally recognized
research program focused on turbulent flow control and
data-driven methods for fluid mechanics.
He's definitely established himself as one of the leading
researchers looking into this. And one of the topics that we
bring on today, of course, is around like explainable AI,
causality, reinforcement learning, reduced order
modelling, and of course the topic of the moment foundation
models for fluid mechanics. He, along with people like Steve
Brunton, has done a lot to really educate the community,
also the public in terms of these fluid mechanics and how
machine learning can be used for.
It's really, he's got some great YouTube videos and
explanations. And what I particularly enjoy
about his work is he doesn't just ask whether AI can make
predictions faster. He's looking at the question,
can AI help us to understand the mechanics of turbulence and to
ultimate accelerate scientific discovery itself.
So, you know, in this discussion, which as with any of
them, is never long enough to fully cover everything, you
know, we dived into some of the bigger questions facing CFD
today. So you know whether fluid
mechanics will have its ChatGPT moment, how close are we to
foundation models can be generalized across different
flow problems, and why he's actually more optimistic about
that possibility than than ever before.
As I mentioned before, the explainable AI causality,
understanding where they come from.
Can you explain how AI is getting to it?
And one of the things that we also look about is focusing on
the role of reinforcement learning in terms of
optimization and control and not just using AI as a pure
predictor. And then towards the end, we, we
sort of zoom out and talk about the future of the field,
agentic AI, autonomous scientific discovery, and also
how universities should be adapting education given that AI
is changing the whole landscape.
So I, I think this is one of these conversations that, you
know, I learned a lot from it and, and I hope you do too.
So sit back and enjoy this episode with Professor Vinuesa.
Thanks very much for agreeing to do this.
You are definitely on the list.
Of people who everyone says, oh, you should be speaking to him.
You know, I read the papers, I see stuff.
And you're the sort of person where when I read the paper, I
was like, yeah, I really should have thought of that.
That's a really good idea. So, yeah, thank you.
No thanks for having me. It's a real pleasure.
And yeah, looking forward to the conversation.
So maybe we could get straight into it, which is the, I guess
the hot conversation now. Which has only got more intense.
Is this foundation models?
You know, can we somehow make a model that could?
You know, compute any fluid flow, whether it's, you know, a
geometry variation, A boundary condition.
Where do you think we are along that journey and how close?
How close do you think we are? To to get in there.
Yeah, I think few years ago I would have probably given a
different answer, but things are changing quickly in a in a
good way. And I think we're getting closer
to having and it all depends on the type of question that you
want to answer, right. What type of predictions and
what level of accuracy do you want to use the systems for
design, Do you want to use the systems for, for scientific
insight? And of course what quantities do
you want to get with what level of accuracy?
So there's many possible questions and directions to go
into. But in general, I think that
we're getting pretty close to having systems that can perform
very well even for quite complicated quantities and quite
some nuanced phenomena on on our system.
Especially because depending on how you define your model, and
depending on the type of question that you may be
interested in answering, you don't really need to mimic all
the physics of the real system.
You might just focus on a subset of questions or a subset of
mechanisms that could be represented encapsulated in a
smart way. And that's basically what your
model does, and not necessarily all the intricate interscale
mechanisms that give rise to those particular phenomena.
So yeah, I think we're getting closer to having systems that
truly can give us the right accuracy, the right performance
in in quite challenging problems.
Yeah. So I'm intrigued to know what's
changed your mind. We said a few years ago you
would have given a different answer.
Can you pinpoint anything that has, yeah, changed over the past
few years that's changed that? There's a couple of things.
The first one is we realised that we could identify those key
mechanisms pretty well in quite complex systems using
causality, explainability. So not just in idealised
versions of the whole flow or in reduced-order representations,
but in really full fidelity high order versions of the system, we
could really interrogate the data, identify what are the
mechanisms that are the most important and focus on those for
control, for modelling. So I think those almost
surprising capabilities of this explainability and causal
frameworks to really characterize the systems has
been one big pillar. And the other one, the
performance of some of the generative deep learning
methods, especially in the last few years, diffusion, flow
matching, conditional latent diffusion, these systems are
really capable of generalizing to an extent that I mean not so
long ago I would say that it was not really possible.
There's still quite some more to do and there needs to be some
smart way of formulating these problems in order to achieve
real generalization, but I would say that that we are really in a
stage where things are pretty impressive.
Maybe you could explain, pardon the pun, what you mean by
explainable AI. Yeah, that's a, that's a good a
good question. So what we mean with the systems
and what constitutes an explanation and even the, the,
you know, the, the grammar and the definition changes a little
bit depending on the, on the field you have explainable AI,
explainable deep learning interpretability.
So there's different kind of flavours to it.
But the way that I use this term explainable deep learning would
be kind of assessing for a particular model what features
of your input matter the most to the prediction of that model.
So really identifying kind of like in a feature attribution
sense, what, what, which ones of those components of the input
really matter to your output. And why is that interest in the
context of high dimensional chaotic engineering systems?
Because if you create a model, an input output model, where I'm
just going to give you an example.
If I have the wing of an aircraft and I really want to
see what flow motions are really affecting the most the drag,
well, then I can create a model which given the flow, it just
predicts the drag on the wing. And that's a predictive model
that hopefully performs reasonably well.
But if I use this explainability methods on that model, then I
can go point by point and really identify which points are
contributing the most to the drag on that wing.
And that idea can really allow us to identify volumes of
importance of importance for that particular question, for
the drag in this case. And what we have found is 2
interesting things. One is by using these
explainability tools, we realize that the classical approaches to
study turbulence and people have been looking at vortices,
streaks, Reynolds stresses, many other quantities, right?
In fact, when people want to show off a bit and they do a
very nice CFD visualisation, very colourful visualisation,
they will show you lambda-2, right, or something similar.
They show you vortices because it's very spectacular and it's
very turbulent. And what we realise with our
methods is that the vortices don't matter so much actually,
at least for the friction and for the drag.
They matter for other things, but not for the stuff that you
are trying to show them for. And the reason is the fact that
you can see something does not mean that it's important.
You simply can't see it. And a lot of the turbulence, a
lot of the turbulence research has been a bit misled by what
you could see in experiments, which of course it has been very
helpful and it has been very illustrative.
But sometimes you need to step away a bit from the physical
intuition and just simply interrogate the data in a more
diagnostic way and let the data tell you what matters and what
doesn't matter. So what we've found is the
classical approaches to turbulence research.
They were only telling a part of the story.
So in different regions, close to the wall, the the Reynolds
stresses were important. Farther away, the streaks were
important. Farther away there were all the
Reynolds stresses that were important.
So the classical views were not wrong because obviously, you
know, there's a lot of time and effort.
They brought it to it. They were just telling part of
the story, right? Like the three blind men and the
elephant. They were, you know, giving
different partial views of what the elephant look like.
But all the classical perspectives together is what
actually comes close to what we identify with our explainability
methods. And why are we so sure that
these methods are giving us something that is interesting
and physical? Because when we devise control
mechanisms, when we manipulate the flow to diminish the
presence of these mechanisms that we identify, that's when we
get the highest drag reduction.
So it's not just that purely we can find nice colourful volumes
of structures is that those structures in a causal way are
actually affecting the drag in this case the most.
And they are tackling those structures is really looking at
the root cause of the drag, so the disease and not the symptom.
And that's why we think that this is really a good way to
look at these mechanisms and not only from kind of controlled
perspective or from a satisfactory knowledge point of
view, also from a modelling point of view, because those
mechanisms are the ones that contain the crucial information
if you want to be a good model for that phenomenon.
So how do you see the the challenge that I still be not
got my head around which is the following.
If we assume that for a foundational model or you know,
surrogate model, we're assuming that we have lots of data of
wings of cars, of buildings, etcetera.
At least today, you know, most of those have, you know, a
decent error to what is the real turbulence, what is the real
flow? And you're asking the model to
learn it. And sometimes the data comes
from different sources. You know, the wing may have been
done with this turbulence model, The car may have done with
this. So how do you see the foundation
model? Is it ultimately yeah.
Do we have to train on DNS? But then we can't train on DNS
because it will just be too expensive.
Do you need some DNS? I still feel like the foundation
model is ultimately going to not be as good as people think.
Yeah, because no, I totally see
your your point. And I think here the key is
going back a bit to the one of the first questions, what do we
want this model for, right? And I mean, I don't think that
we are trying to build these models to replace DNS, right?
I mean we're not going to because that computationally
would not really make sense, right?
So I think that we're trying to build these models to either
accelerate design and optimization and or achieve some
sort of scientific insight in our system.
So how can we handle data from different fidelities and
different modalities? That's something that in what
we're building in my group, we're trying to embed the
uncertainty quantification with active learning loops in a
sense that we know that not all all our data is of the same
fidelity. When we are trying to generalize
and we're trying to produce data for new cases, the system will
tell us, look for this particular case that you're
trying to produce data in some latent representation, your
uncertainty is very high. So you should probably go back
and if you know, if you're trying to do a helicopters,
well, your helicopter data is actually pretty bad.
So you should be there, have higher fidelity or, you know,
run more experiments or really improve your model there.
And I think that's one strength actually that we can
progressively keep improving our model as new data becomes
available. Now your data for helicopters is
pretty bad for what, right, Because it depends on what
you're trying to do. And here is when we need to be a
bit a bit realistic about the metrics and about the targets
that we want to hit. So if we are trying to get lift
and drag, right, that's one type of accuracy and one type of
model. If we're trying to get the
spectrum right and all the interscale mechanisms, right,
then that's a different type of model, right?
So I think in that sense, depending on the type of
approach and the type of figurative that we want to
achieve, we may have to resort to different data sets and
different ways of assessing the error and the uncertainty of
our systems. Yeah, no, I, I agree.
I think the there'll be probably one side of the community that
happily uses, you know, Reynolds averaged or even panel methods
or or or lower and are, you know, quite comfortable with
models that are not of DNS accuracy.
But yeah, yeah. On the other hand, I, I, I can
imagine just as companies spend a fortune constantly trying to
get to higher and higher fidelity CFD that maybe the
models will, you know, go with it.
One question, because I think some of your background, you
know, was on, I guess, would it be fair to say like reduced
order modelling and and some of that side 2, maybe a more like
lay audience? I'm not saying everyone
listening to this is a lay audience.
That's quite the opposite. But there are some people who
are maybe not as grounded in all the latest common question I
get is, well, machine learning is just reduced order
modelling. How would you chart the
evolution, you know, from like what people would call ROMs
before to machine learning? And what is it about modern
machine learning that separates it from reduced-order modelling?
Unless you classify them as to
say. Yeah.
No, that's a good question. Depends a little bit on what
you're trying to do with your machine learning, right, Because
also machine learning is a quite broad area.
So, I mean, and we have some work that we've done before on
machine learning for CFD. So how should you or what are
the areas where machine learning can help CFD?
And this is work that I did with my friend Steve Brunton some
years ago where we found three areas.
We found accelerate DNS, improve models, both RANS and
LES and improve reduced-order models or surrogate models.
And, and, and, and that's if we are thinking of machine
learning for CFD, something that we made quite clear at the
time, this is 4 years ago was that machine learning in
principle would not replace CFD.
It's not about, and I think that many people when they think
machine learning for CFD, they're thinking automatically
replacing CFD, right. I, I don't think that it's
really about replacing CFD, but rather complementing and
helping. So reduced-order modelling would
be just one area within all the spectrum of possibilities
for machine learning can help just within CFD, but we can also
think about control and optimization where this
fantastic reinforcement learning work, for example, on really
finding new controller strategies.
So it's very broad, but I would say that if we focus on on
surrogate models, traditional models, one key has been the
possibility of handling well, nonlinearities in your in your
model development, right in your surrogate.
And of course POD, DMD, I mean, they're mostly linear, although
you can have nonlinearities in there.
But having neural network based compression at scale, which is
what you can have now with modern autoencoder architectures
that really allows you to have very impressive compression
rates, right. So quite some capability to to
distill the essential physics of your system.
And something that we found also in some of our work is in, I
mean, autoencoders are good at compressing, right?
Compressing anything, images of cats and dogs and images of
turbulent flows if you wish. But turbulent flows, they're
images, their flow fields, they contain much more information.
They're much richer than cats and dogs, right?
There's a spectrum, there's a range of scales.
So you should somehow, when you compress, you should somehow
disentangle the latent representation, because if you
do that and you can use beta- VAEs, you can use hierarchical
priors. There's different ways to do
that disentanglement, but that really allows you to encapsulate
in different latent representations and different
latent vectors, different physical phenomena of your
problem. And that's going to be more
effective from the interpretability point of view
of your surrogate system, but also from a predictive point of
view. If you want to make predictions
in in the latent space, that disentanglement is actually
going to be quite, quite helpful.
So, yeah, I mean, I guess it was a bit of a long answer to your
question, but machine learning is not just reduced-order
modelling. Let's say that there's many
other things that one can do and within reduced-order modelling
that, you know, capability of really having autoencoders at
scale and disentangling the latent representations has been
quite critical. And also Transformers, that's
another, well dimension let's say, or another topic, which
also has helped in building the temporal dynamics of these
reduced-order systems. I mean, where do you see?
At least my impression has been that even though the turbulent
modelling started off as being one of the the big focuses of
the fluids community, it does seem to have slightly petered
out a little bit and the focus has shifted a little bit more to
the surrogate modelling, to the sort of reduced-order
modelling. Is that something that you've
also noticed? And do you think that's just
because of a lack of progress? Do you think it's because the,
you know, the improvements? I'm just wondering if how you've
seen that? Yeah, no, that's another good
point. I think that so turbulence
modelling can be done in different ways.
And One Direction that has been adopted by many people has been
to start where more classical approaches to turbulence model
modelling start. Basically to which in a way is
just making an empirical assumption of how the, you know,
the Reynolds stresses need to behave.
To some extent. There is some empirical
assumption, more or less sophisticated and then try to
use machine learning systems to continue from there.
So in, in other words, to fit coefficients on classical
models, right? And well, that's not maybe the
most revolutionary thing, right? Because if you do that, and
that's what many people have done.
And I think that's why a bit of the ML community could have got
a bit disappointed with ML for fluids, because they were using
ML to adjust coefficients in two turbulence models that have
been around for decades and which have based on assumptions
that are a bit crude, perhaps, right.
But for a reason, they were crude because there were not
other approaches to, to modelling that were, you know,
computationally possible at that time.
So I think that could be why there has been a bit of slow
down. But at the same time, I've seen
some promising approaches. I mean, I'm, I'm a bit of a fan
of reinforcement learning. I've, I've been developing many
methods within reinforcement learning for control, for
optimization and also for modelling.
We have some, some projects on reinforcement learning for
modelling and there's other groups who are doing great
things in this space. And I think that the idea is why
start with a model that has been around for, you know, 20305060
years and try to adjust those coefficients when we can maybe
take one step back. So instead of asking my machine
learning model to say for the Smagorinsky model, what should
be my coefficient, Can you find it to, to, or even say to my
machine learning model or just find the subgrid-scale tensor to
be able to fit the statistics of this this particular channel
or particular flow? Maybe we can be a bit more
abstract and say, look, I don't know what the mean flow or the
fluctuations will look like in this particular case.
I, I don't know what happened. I've never run this wing or this
aircraft, but what I know is that turbulence is characterized
by certain energy transfer mechanisms across scales, and
that energy goes in this direction and there is an
inverse cascade. And there's certain phenomena
that need to be true from a spectral point of view.
And from a mechanistic point of view, whatever your model is
and whatever the forcing term that you get each step from
your reinforcement learning or whatever optimizer that you're
using, just respect those physical constraints.
Because my flow needs to be physically correct.
And physically correct does not mean this is your mean flow.
You need to adjust to this mean flow because that's a circular
argument, then you need the mean flow, right?
But rather not only a circular argument, it's some more, it's a
much more constraining goal to fit the statistics right.
But something a bit more flexible, such as making sure
that the energy fluxes are correct step by step is
something that is less impose imposing.
And it's also something that is a bit more manageable for a for
a system for an optimization system to to be able to adapt
step by step such that those fluxes are OK.
So I think the second direction of making statements that are a
bit more general and still correct from a physical point of
view, that could be more promising to develop more.
I mean, one of the things that I've started contemplating, and
I'm interested to see if you've been the same is for for a
while I was only thinking about, you know, the machine
learning, either in the context of learning what the turbulent
viscosity should be or, or more broadly, what a surrogate
model, you know, should be in terms of a transformer based
or some neural operator or, or whatever.
And the criticism has been that it's, how can it match the, the
brains, you know, the thought process that, that, that we've
had. And I saw at an event, I was at
the British Library, there was an AI, somebody organized UKTC,
the Turbulence Consortium in the UK.
And it was Luca Magri from Imperial was organizing with
some people. And there was a Professor,
Michael Leschziner, who, you know, is a sort of pioneer of
turbulence modelling developed with Brian Launder, you know,
all these Reynolds-stress models.
And, and he rightly said to me that he was, you know, gently
wondering how could these AI models, you know, know, all
these things that, that, that they've done.
And then I started to think, is this really, and I don't know
if the capabilities today where the agentic thing comes in
because in some ways is it more realistic to ask an LLM to
reason through the process of how you would develop a
turbulence model and have links in to software tools to
explore it and go through that reasoning process and
reinforcement learning. Then just say, hey, try and find
me the the latent representation of this.
Do you know what I mean? Do you think almost the
scientific discovery that piece is the better route rather than
just expecting machine that is?
So I don't know if I've explained that, but.
That makes perfect sense. I think it's, it's a very fair
question and we are trying to well to really first of all
understand the potential of agentic systems and 2nd, we
are learning to ask the right questions to these systems.
I think before getting into that, I think that a preliminary
point is what is the right way to represent the data that we as
engineers and scientists in you know, higher order chaotic
complex systems. What is the right, the right way
to represent the data and such an exploration from these agents
or whatever or, or any scientist, what's the right
platform to represent the data, right?
Because there's quite some work on LLMs to do this and to try to
achieve discovery, whatever we define by discovery.
But you know, if we think of the of turbulent flows, of course,
this is these are systems that are having the broadband in
terms of the spectrum, they have multiple scales, multiple energy
fluxes. Perhaps text is not the right
way to represent this very complex data, right?
And perhaps using LLMs directly and text as a platform to try to
explore complex questions in this space might not be the the
most suitable approach. And what we have been thinking
and exploring is what if we can create latent representations
that are particularly attuned to represent important properties
of these turbulent-flow systems. And, and that's an approach that
we have been developing in our group where we are trying to
build a foundation models, kind of latent representations of
very broad ranges of cases such that we can use those latent
spaces to explore the, the, you know, the, the complexity of
these, of these turbulent flows. One guiding principle has been
information, information fluxes, causality.
So what if we can create latent spaces where causal relations
are maximized or where the disentanglement of the variables
is such that I can very easily explore physical mechanisms
through that latent space. And I believe and again, this is
stuff that we are very excited about because we are we're
developing it with pretty promising results that using a
agentic systems in that latent representation might be a good
way to achieve discovery. So in, in other words, I have a
data from wings aircraft flows in cities a compressors and
these are very different fluid mechanics problems.
But if we find smart way to compress all that data and
express it in the same latent representation, so now suddenly
I don't have a wing and a city, but everything is expressed in
the same language on the same table.
And then I can allow this agentic systems to explore this
latent space very efficiently. Because of course this AI
systems can't really find one side compress everything so
much. I can find patterns across cases
that might not be obvious in in the physical space for us
humans and neither for agents, right.
Because finding a connection between a wing and a city, I
mean, yeah, maybe if I look like that, I see a vortex that
looks similar. But you know, it's, it's a bit
more anecdotal, anecdotal than anything.
But when I express everything in that space and then I can do
that automatic and autonomous exploration with these agents,
then I can try to get insight. Then I can try to find causal
relations that are key in that latent space.
And then our job with explainability tools is to
express that insight in the latent space, which is
completely non understandable for us back in the physical
space and say, look, this is what insight looks like in the
latent space. That means that this vortex from
the shear layer of the wing is actually very similar to this
vortex from the shear layer in the separation of that building
in the city. So that's actually a common
mechanism. That's interesting.
And I can find that through compressing, expressing in the
common space, A autonomously exploring and then going back to
the physical representation. Yeah, that's, I really like the
I've you've hit the really good explanation there because this
is something I've been thinking in my head is and I I'm not, I
need to look into it more to be completely honest.
But I almost see 2 slightly different approaches
traditionally, which is 1, which is kind of what we wrote in
this paper. We put out recently this idea
that, oh, you, you know, we're going to run lots of simulations
of planes and cars and cities and turbo machineries and
basically every single possible geometry and boundary condition
that we see in real life, which feels in some ways the more
logical and achievable way. It's just like you just have to
have a lot of them. But then the other side is that
actually it doesn't matter whether it's a plane or a city
or whatever, It's just you're just trying to learn the flow
physics. And actually you may massively
have way too much information by trying to do all possible
ones. Which then I thought, couldn't
you achieve this? Do you really need wings?
Couldn't you just create lots of random shapes with lots of
random boundary conditions? And as long as you take every
possible consequence of shear flow and pressure gradients, it
doesn't actually matter if it looks like a plane because
you're just trying to learn the physics.
That's a very good question, I would say.
I mean, that can be helpful because what you want is to be
able to explore such a latent representation very broadly, you
know, and that you can really investigate many configurations
and calculations and so on. At the end, well, we want to
revert back to something that we can understand, right?
And then probably having some random shapes with some random
pressure gradient and curvature distributions would be helpful
for the exploration because that would allow us to identify
different mechanisms that probably we haven't seen just
because they were not ideal dynamic, but maybe they were
helpful for other things. But probably in our data sets,
we want to have shapes that we're familiar with and
applications that we're familiar with just for the kind of
interpretation point of view, so such that we can connect it.
But from the latent space generation, yes, the most the
more diverse and the more crazy, the better.
In fact, what what we are doing, what we are building is systems
that the agent not only does the exploration but also
generates new geometries by itself.
So we have this problem that we like very much and we're having
a pre print coming out very soon on this.
So it's basically two cylinders, to the keep it simple for now,
but it's 2 cylinders with different radii and different
separations. And we just want the our system
agentic AI with a foundation model system to to explore the
wake of this tandem cylinder arrangement and just look at the
recovery of the wake. And then what this does is,
well, in order to do that, and this is trained on few cases,
right? It's not trained on many cases,
but I need to explore the whole space, right?
So these agents, what they do is, oh, but I need to look at
this configuration and calculate my wake characteristics and
internal quantities and you know, displacement thickness,
momentum thickness, okay, I know this.
But then I need to look at a different case and a different
case and a different case. And then it starts to
sequentially understand what the scaling of the wake recovery
looks like as a function of the parameters of your problem.
And you haven't not had to create hundreds of cases.
You only have to create a few cases.
But it's the foundation model combined with the agent that is
doing that autonomously is doing the discovery for you by
identifying what regions are important and creating and
generating new data in those regions.
In those regions. I think that's where we are
going so that we can even find a discovery of cases that we don't
even think of because, because we have been looking at patterns
influenced by the past history, right?
And kind of like the need of applications, but autonomously
exploring the whole space as agents can do.
They can really multiply the the the insights and new solutions
and new mechanisms that we can actually discover.
And and that's why I'd feel and I'm guilty of this myself, that
we are still probably too, too much splitting the fluids task
or the data generation task from the machine learning task.
And so, you know, I'll be transparent for the data sets
that I've typically generated, they have been kind of segmented
a little bit for practical reasons, for people reasons.
So you're like, right, Well, you know, you have a bunch of
meetings, you decide what you're going to do.
You then kick off the data generation exercise, you
generate lots of data, and then you train the model.
And where, as you say, what you really want to be doing is
constantly them being in a loop together and only generating the
data it needs to generate. But I don't know if you found
this in your work, but one of the problems is that even the
machine learning frameworks are typically not even written in
the same language that the data generation is.
And doing that on a big HPC machine and training and
translating information, it's, it's quite challenging and maybe
brings the topic that, you know, we briefly discussed
before we started recording, which is this fluid community,
computer science, ML community.
How have you seen those two communities come closer?
Still not close enough, You know, how how have you assessed
this? And, and, and to be fair, you
have done a great deal. And to be fair with people like
who you collaborate, Steve Brunton of of trying to, you
know, bring a little bit together.
But yeah, how how do you see those two communities at the
moment? Well, thank you.
Thank you so much. First, I think the problem that
has happened for a while and still present, although probably
getting a bit better, is that they have been a bit
disconnected in really understanding the the depths of
the problems in fluid mechanics.
And perhaps the fluid mechanics community, when trying to adopt
ML methods, has been sometimes a bit naive in just.
Adopting very vanilla architectures and maybe giving
up a bit too quickly without fully understanding.
So, you know, if the mechanics community is guilty in part, and
I think what has happened also from the CS community is that,
you know, I mean aerodynamics from I am at the aerospace
engineering department right here in Michigan.
We train engineers for many years to understand the
mechanics, aerodynamics, structures and many other
things. But of course fluid mechanics is
a big part of it. So you can't understand all the
complexities of fluid mechanics, you know, in the
course of 2-3 months, right? Because there is a lot to look
at. These are very entangled
problems and sometimes the questions and the problems that
we have in fluid mechanics are not what typically the CS
community has looked at necessarily.
Because you know, the type of data that has been used in the
CS community to develop methods is not necessarily of the same
characteristics as the fluid mechanics data.
And this is very rich data with a very broadband spectrum,
multi- scale. This is really, really complex
physics, right? So maybe looking at the MSE is
not the right metric, right? Because this only tells you a
very, very partial view of the story.
And maybe even having a model that predicts the MSE, well,
that might be useless in some cases, right?
We maybe want other things. Maybe we just want to look at,
you know, systems that predict separation, right or that
predict certain aspects of the mixing or maybe, you know,
certain structures, certain mechanisms as I was mentioning
before for, you know, for a drag producing mechanisms.
So it could be that for my purpose, if I want to reduce
drag, having a good MSE is actually counterproductive
because the MSE is going to drive me to certain events,
which may be very energetic and they may be contributing a lot
to the MSE, but not so relevant towards the drag generation,
which is the stuff that I want to, you know, focus on in my
application. Therefore, a model with a much
worse MSE but getting the error right in the right mechanisms is
way better, right? So I think the this type of this
type of interaction is what is really going to help us build
more useful models, at least from the fluid mechanics
community. And of course, this is not just
for fluid mechanics. This is applicable to any area
of, of science, right? But of course, I can speak, you
know, with more depth on the fluid mechanics problems because
what you want, and this is another question, you know,
let's build a foundation model. Let's build a big ML system for
fluids, OK. But for fluids for what, right.
I mean, within fluids and within
aerodynamics and within combustion, I mean, there are so
many questions that we can look at and probably the model that
you build is going to be quite different for all those
questions, right? So I think that's really
understanding in more depth what the fluid mechanics problems
require. What are the questions that we
have? What are the questions that we
don't have? How to build models that can
really effectively tackle those questions, that can be a much
more effective use of everybody's time, basically.
Where do you sit on the debate around a data-driven versus
physics driven models? You know, again, the usual
criticism from people who are maybe from a fluid background
is, you know, any of these data-driven approaches cannot
guarantee mass conservation, energy conservation.
They can't guarantee, you know, we should be implicitly or
explicitly imposing, which I guess is where some of the
PINNs, you know, things started.
But then the successes seem to have been not as strong in the
sort of physics informed, inspired, conditioned.
So yeah, where where do you sit on that side of the fence?
That's, well, that's, that's a big question, right?
Of course you need to, you need to use any physical information
that you have about your system when building your models,
right. But if you just use the
equations and nothing else, then you're building a numerical
solver, right, with a data-driven numerical solver.
And if you just use data, then you are a problem, you know, be
being very wasteful because there's a lot of information
that you have about your system that you need to relearn.
Like, you know that your system has certain symmetries, certain
conservation properties. If you don't embed that in your
system, well, first of all, your model will be wrong because
you will never be, you know, conservative exactly right or as
exactly as possible. But second, even if it's a very,
it's a system that is very conservative, you have to learn
that right. So you need to use data and
compute to get to that point. And of course, that's something
that you knew from the beginning.
So it's not very smart to to waste that information.
So like with everything in life, right, no extreme is going to be
optimal. And I think that both extremes
have been explored and we have learned a lot from it.
To me, being able to use symmetries, conservation laws in
your problems. If you're trying to solve for
several velocity components, I mean, do you know that there is
the flow is incompressible, Do you know that you can use
incompressibility to get the velocity component or use that
in the way that you're building your system?
But also going back to an idea that I like very much, this
whole explainability, causality mechanisms.
If we know these things, why don't we try to build systems
that are really focused on this? And one example is, let's
imagine that I want to create a system that predicts very well
the acoustic field of an airfoil or a drone, right?
Because drones can be very noisy.
I want to be able to control the acoustic field, the noise
produced by this drone. Why don't I use some of these
causality explainability methods to identify and pinpoint the
mechanisms producing the noise? And by mechanisms, I'm talking
about flow regions interacting with each other in physical
space. And then when I build a
predictive model, I don't look at the, you know, trying to get
neither just the governing equations like in a PINN way or
just use data to predict the flow fields.
But rather, why don't I try to predict just those structures,
those mechanisms that produce the acoustic field in that way,
you're really encapsulating in those structures a lot of
information. And that's where I think that
one can have computational savings with respect to, you
know, doing a much more resolved simulation.
Because those structures producing the acoustic field are
the result of a lot of, you know, interscale energy
interactions that you need to resolve when you do a DNS.
But the model, I mean, once that DNS is run, those structures
that are producing the acoustic field and so on, that the result
of those interactions, right? So you don't need to re simulate
them all the time. Once you know what the
mechanisms are, you can try to develop models that target those
physical mechanisms precisely and not everything else.
Of course. That's why I repeat, sometimes I
want a model for what? If your model is for the
acoustic field, then my advice would be, well, identify what
are the mechanisms producing the acoustic field and try to get
those very well, right? Because probably you will not
get a perfect model, but you will get something that is
pretty accurate and pretty efficient because you don't need
to solve everything. But if your goal is to have
something that gets the acoustic field very well and the gusts
and the turbulence and the heat transfer well, then you need a
multi physics DNS, right? Then, then you need to solve
everything. So it depends a little bit on
what you want and of course, embedding that physical insight
in your model, not as equations, but as mechanisms
that you can then, you know, recreate with your system.
I think that that's a pretty promising way to go.
And how from a, you know, you're, you're obviously a
professor at university. What do you think?
What's the role of academia in this, focusing more on like the
students in the education side, you know, do you try and team up
more with the computer science departments?
Is it a matter of swapping people over like, but but then
equally, I guess you don't want to not teach some of your
existing content. So how do you balance getting
rid of some content, bringing new ones in?
That's that's a key question. And we're having many
discussions around this. For example, here in Michigan, I
mean, we're having a lot of conversations around our new
curriculum and how to well adapt to AI era, LLM-based era.
So we're having many conversations around this.
So first of all, we have a very good interaction with Los Alamos
National Lab. We actually have a bunch of Los
Alamos scientists sitting on our campus and working with them
daily and interacting with them constantly.
So we really get access to that synergy and that cooperation
with computer scientists, which are world class, right?
So with with really incredible computational facilities.
So I think such an interface and such a close connection is, is
essential. But the other thing is from the
educational point of view, we need to be of course aware of
the fact that students have access to LLMs, right?
And and that, you know, the way that we teach, the way that we
assess is different from what it used to be.
But in a way, and this is conversations that we're having
with my colleagues and I think that there's quite some
interesting, I'm promising directions in here.
We, we don't need to necessarily see this as a disadvantage, but
rather as an opportunity to use these systems to encourage
students for more exploration, for deeper, for, for deeper
questions and deeper assessments of the content almost in a
customized way, right? So, so I think that rather than
thinking that students are going to be lazy and just look
up their answers and not think, rather use these these
systems to help students think in a way.
But do you feel that? I guess flicking around the
other way, do do computer science students need to learn
more about the sciences? You know, there's a lot of like
AI for science. If the engineers are all
learning some of the computer science being, then what are the
computer scientists doing? Is there, is there an equal
thing on the computer science say hey you all need to be
learning some science domain because AI is going to do the
some of the core CS stuff, you know?
I was, well, I'm, I can tell you that there is, I mean, there
is an increase of enrollment of students in, in areas like
aerospace engineering and, and part of it comes from computer
science, although of course the computer science degrees,
they've been excellent at adapting to, to new technologies
and new well, developments within AI for science.
But there is more and more overlap and more and more, you
know, joint educational programs across systems and across
engineering disciplines. And yes, I would, I would agree
completely that if computer scientists are going to be the
developing AI for science or AI for engineering methods, then
they're going to need more background in these, in these
topics. And that's something that is
really that's multidisciplinary wealth and and richness is
what's going to bring really stellar research in the next
years, right? That 1 is really capable of
finding completely new directions like the foundation
model and the agent are finding patterns across data sets that
you would not imagine, right. When we start to interact in
disciplines at a different level, we're going to find such
connections that maybe we did not expect at the beginning.
So one of the things that, you know, you mentioned a few times
is on the like control and optimization side of things.
And I feel at least myself a little bit guilty that I, I'm
always thinking of, you know, the, the impact of machine
learning or reduced order models in terms of a predicted
capability. You know, we can predict this
flow much faster. But I guess for industry or even
for scientific discovery, but particularly for industry, it's
almost always a kind of optimization problem in a way
isn't it's like I need to have the best design or the lowest
drag or the it's never just a prediction.
So, So what progress have you seen with the sort of automatic
differentiation, the the ability of machine learning to maybe
help advance that optimization process?
Yeah, I I think that prediction is important.
It's interesting. It's not the most impactful
application of machine learning.
It's in optimization and control.
That's where we can really have the, the biggest, the biggest
impact. So as I mentioned before, I, I
worked a lot on reinforcement learning.
So that's an area where we have been able to control cases that
were not possible three or four years ago or at least with this
degree of control authority. And not only.
So I would say not just from a purely design point of view of
reducing drag or enhancing mixing.
I would argue that for discovery, for really
understanding the mechanisms and the building blocks of these
physical systems, you can use optimization and, and, and
reinforcement learning. For example, I mean, imagine a
turbulent flow where a made-up turbulent flow where the energy
fluxes are restricted to certain scales and to certain
wave numbers dynamically, right That you do it as the flow
is running and, and, and you want to do that in physical
space. So you want to really affect
certain structures with a certain forcing.
Well, you are probably going to have to do it with some
optimization technique. And reinforcement learning is
good for optimizing systems that are changing dynamically.
So you you're going to end up with a with a modified system or
a system where production is minimized and dissipation is
maximized. How does a flow like that look
like? And let me study, let me study
that new flow, right? I mean, I have created a
completely made-up flow thanks to optimization.
And now I can look at the Spectra, I can look at the
structures and learn something about my problem, right.
So yes, these capabilities with optimization are key also for
discovery and so for getting insight.
And going back to your previous point about how new systems and
differentiable solvers, I mean, we, we are really having the
chance of solving problems at scale of this optimization type
of system, which was not possible before.
And inverse problems where we can, you know, optimal sensing,
we can really look at optimal initial conditions and optimal
perturbations really for very, very challenging configurations.
All of that is possible now thanks to the scale that we can
achieve with these systems. So I think that's, yeah, I mean,
prediction is, is good. It's interesting.
It's only part of the problem. It's really in the control and
optimization and in the explainability of those
predictions. Like, OK, I have predicted very
well in my system, but why are those predictions happening?
And what are the really important physics and effects
that I can observe with my system?
That's that's the stuff that I think is is is actually
valuable. So maybe a sort of final
question or, or topic looking towards the future a little bit.
You, you said right at the beginning that if, if I had
asked you a few years ago, you'd have said that we're, we're
actually, you know, further away.
So if you look forward in time and let's say to 2030, so 4
years out, yeah, four years out, where do you think we'll be
at? Do you perceive there being a a
sort of ChatGPT moment in fluid or do you think it'll be just
more incremental improvements? Yeah, that's a tough question.
I think one key to go towards ChatGPT moment in fluids was to
realise that we need a good latent representations for our
problem. And I think that now there's
several groups of several people working in that direction.
So I think we're probably in the right path.
I would not. So I don't know if we want to
aim at being having a ChatGPT situation in fluids, maybe
something beyond maybe something more than what ChatGPT can can
do. What I think that we are aiming
at, and I don't know if that would be in 2030, but I think
that it's actually reasonable to think that it should happen in
the next years, is more autonomous discovery, a more
automatic discovery. So one, I like the, the one
example of how big changes in in the paradigm of physics have
taken place and of course, well, relativity departing from more
classical mechanics. I mean, it takes, if you think
about it, a lot of a lot of coincidences to line up in order
for that to happen. And you need someone who is
smart enough and also crazy enough to propose something very
different. And of course, Einstein lacked
all the math background to be able to, you know, develop the
theory of relativity properly. So he had to learn the math and
interact with the right people to be able to do this properly.
And then, you know, to have the
right experiments of the of the solar eclipse to really look at
the deviation of the sunbeams. And there's some fun stories
about how those measurements took place because there were
several groups doing the experiments at the same time,
and they had different types of problems to really get the
measurements that eventually corroborated that light was, you
know, displaced by the sun in a way consistent with relativity.
There's a lot of serendipity in discovery, a lot of
coincidences lined up, and a lot of human trial and error to
achieve truly transformative discovery in science.
And I think that the sort of systems where we can line up on
the same space, the data from different disciplines and having
agents systematically probing and analysing these data sets,
that can really be the way in which we can accelerate that.
But it's this acceleration of discovery, which is something
that, you know, many people are talking about with different
meanings. And I think it's a little bit
shallow the way that it's been portrayed sometimes.
To me, what it means is being able to probe the data and find
patterns in a more systematic way than what it takes us as
humans to take those leaps. Because we're it's really
decades and decades and centuries to line up all those
coincidences necessary for that leap to happen.
And if we can do this in a way that is much more autonomous and
much more systematic in that sense, that can be accelerated,
right? And this sort of systems I, you
know, I think that we are really getting there actually.
And, and, and this is not really about replacing human insight
or human input. This is really about
accelerating the exploration on the capability of coming up
with mechanisms with hypothesis with a really patterns that
are non trivial across across disciplines and across cases.
So yeah, I think that we're heading in that direction of
really being able to make autonomous discovery thanks to
the possibility of compressing data and interrogating it
systematically in a way. Yeah, it does seem that there is
this, I would say inflection point now where I would largely
agree with you, you know, a few years.
Ago when I. Started to think about some of
like a foundational model, you know, it it did for various
reasons just felt quite Yeah, just like impossible almost.
It would, you know, may still be, but I I also get the feeling
that there is a, a general movement of replicating the sort
of ChatGPT success, you know, the the set.
And this is broader than just fluids across science, across
physics, across all, you know, possible disciplines that, you
know, it worked for that which and, and I'm not I haven't
studied a a great deal, but I, you know, I know enough to know
that many people thought that it couldn't be possible, right.
You know, people weren't saying, oh, ChatGPT Oh yeah, we knew
that was going to come. You know, it was quite a big
impact and, and it, it was done at a scale that was larger than
anyone thought possible. And I, I do have a feel that
made people more ambitious, you know, and, and think, well, if,
if it could work for that, it, it could work.
But I do like your argument on that.
Sometimes you need to look at things a bit differently because
I have a funny feeling that the approach that ultimately makes
it may not be the one that is mainstream today.
You know that that like in all in science, there's always, it's
always the person who takes a little bit of a, an odd
direction at the time which turns out to actually be the
one, you know? An agent to come up with that
direction, maybe. But that's what I meant.
Like the, I can totally see how with humans sort of in the
loop, you know, in, in, in terms of guiding things, that the
maybe it is ultimately the power of ChatGPT to find the next
ChatGPT. You know, that those LLM
technology and the agents and the way they work together with,
of course, the way that you structure the data, you know,
may help. And, and then maybe that's
artificial general intelligence.
And then we can all just, you know, go and retire and, and,
and chill out. But yeah, I really appreciate
having the chance to, to, to chat about this.
What I'm going to do for people listening or watching is to put
a bunch of links into the, the comments because there's a few
papers that you alluded to that are really good, you know, great
reads that talk in more detail about some of that explainable
AI and some of the work that, you know, you've done.
So I'll, we'll put them in the link and highly recommend that
people read through them to go even deeper than we discussed
now. But yeah, I, I really appreciate
it. And I think your work is going
to be seen as a major contributing factor to hopefully
achieving that ChatGPT moment. Well, thank you very much.
I appreciate it. Maybe that's a bit optimistic,
the money work, but I appreciate that much.
And at the end we, we have fun with what we do.
I think that's kind of like the key, the key idea and we keep
learning. So that's that's pretty much it.
Exactly, exactly.
Great. Thank you so much.