Hi, and welcome to the Neil Ashton Podcast.
In each episode, we explain some of the fascinating ways
that science and engineering are changing the world around us.
We talk to leading engineers from elite level sports like
cycling and Formula One to some of the world's top academics to
understand how fluid dynamics, machine learning and
supercomputing are bringing in a new era of discovery.
We also hear some of their life stories, their career advice,
the lessons they've learned on the way that I hope will be
helpful to you too. So sit back and enjoy this
episode. Hi, and welcome back to the Neil
Ashton Podcast. So today I'm speaking with
Professor Mike Giles, somebody who has had an enormous
influence on the CFD community, but more broadly in, in maths,
in high performance computing. And someone that, as I say to to
Mike at the beginning of the of the chat is his name often
comes up when I speak to, to other people.
And I was always really intrigued to, to speak to him,
particularly because when I was at Oxford myself, he was always
a figure that was, that was mentioned and was influential in
so many bits. Probably for most people, he is
most known as the, I guess you'd call it the lead developer
instigator of the Hydra Rolls-Royce CFD code that is
used still used today by I would assume, thousands of engineers
around the world to design the the jet engines that they
produce. And, and I think what makes him
really interesting is his pivot also to then work in finance,
computational maths as applied to, to, to, to finance, where
he has equally made an impression.
And that's what's incredible for someone to make an impression in
two quite different fields, CFD and and finance being quite
different shows the level of the person, the intellect, the
the sort of innovation potential to switch to a
different field and then still make an impact.
Now I only come from the first field, so my appreciation of the
second in the sort of quant side of the world.
But I've read enough to know and you only have to look at do a
quick Google search to see that he's his work on what's called
that multilevel Monte Carlo methods has been very impactful
in that community. And the third thing that I guess
he has really pioneered in some ways, and one one was one of
the early adopters for is on the high, high performance
computing side and particularly around GPUs.
I should state obviously for transparency, I do now work at
that company NVIDIA, but this conversation was in no way
arranged to to promote NVIDIA. This was organized completely
separately. And it just happens to be that I
work there. So you'll hear some mentions of
it, but please trust me, this is not some sort of, you know,
product placement. And and he was actually working
on this and I think he said that he was maybe the number 2 or
like the second ever CUDA fellow and was looking at this in
2006, 2007. You know, well before sort of
everybody knows the name of NVIDIA and GPUs.
So he, he is always, and he says in the discussion, had an
interest in high performance computers throughout the time
and, and still today teaches classes and, and programs and
does everything, which is, I really love that when you see
someone who's gone through their whole career and it's
completely fine if you do change.
And some people as they progress, become more senior,
they get less hands on, you know, and then they're more
about enacting a vision for what they want to do.
And there's nothing wrong with that.
But I always have a special appreciation for, for people who
are, you know, one of the world's leading professors and
they're still hands-on at the keyboard.
So yeah. Mike is a professor of numerical
analysis at the Maths Institute at the University of
Oxford. As we mentioned, beautiful
building, lovely location. I'm very jealous.
And he was at the University of Cambridge, where he read as an
undergraduate. And we mentioned it, he was a
senior Wrangler, which if you look it up on Wikipedia,
basically means the person who graduated top of class for the
whole university in terms of maths as an undergraduate.
And then he went to MIT, he was Kennedy Scholar, taught there,
came back to Oxford. Now we go through all of that,
but one of the things that's probably worth mentioning and
congratulating him on is that actually very recently he was
elected a fellow of the Royal Society, which is one of the
highest honours that can be bestowed on somebody.
So he's had an amazing career, still has an amazing career.
And like any of these episodes, when I talk to someone like
that, there are so many things that I realized I didn't ask him
after I finished the episode. And I don't think we would have
time anyway to, to to go through stuff.
I will just note that if you Google his name and go to his
personal website, I'll put it in the chat for the YouTube side
of things. He has a great link to lots of
presentations courses that he's done.
So I would definitely look there to, yeah, to find out
more. But yeah, I really hope you
enjoy this conversation. I genuinely did.
I hope you can see it from my face if you're watching it.
I was learning and, you know, interested throughout the whole
2 hours. So yeah, please sit back and
enjoy this episode with Professor Mike Giles.
Thank you very much. I really appreciate it.
As I said just before your name comes up a lot in people I
speak to in the in the CFD and HPC world.
And so I was, yeah, really wanting to speak to you to find
out a little bit more how this, you know, connections started
out. But maybe we could start, you
know, I guess towards towards the beginning.
Now you're a professor at, you know, one of the top
universities in the in the in the world and maths department,
but obviously with an engineering background.
Did you always want? Was your interest in maths?
Engineering? Were you a person with planes
and cars? More reading?
Physics What? What was your early days like?
So I would say in school my interests were maths and
physics. And so I actually, you know,
went to Cambridge to study maths, intending after the first
year to transfer into theoretical physics.
But then, you know, enjoyed the maths, stayed, stayed in the
maths. So, so as a child, that's kind
of the direction I suppose I saw myself going in, but I also
wasn't sure, you know, what I would want to do after my
studies. And so in terms of CFD.
I mean really the, the pivotal thing for me was the fact that
in in going to Cambridge in those days, you did the
entrance exam in November, December and then you had from
January to October to do something else.
And some people have travelled then in my case, I went and
worked at Rolls-Royce for that period.
And that, that was really to learn well, well, to see what
engineering was like, see if that's something which
interested me. So at age 18, I was a
Rolls-Royce undergraduate engineering apprentice.
That was my, my job title. And normally they wouldn't,
wouldn't have taken a mathematician on, you know,
usually it, it was engineers, maybe people in materials.
But I was actually third generation Rolls-Royce.
My, my mother was a programmer before I was born.
She was a programmer with Rolls Royce in Derby.
And my grandfather worked, worked for the company for more
than 25 years in Glasgow. So back in those days, you know,
the application forms asked if you had any family members
working in the company. I mean, these days that, that
that would be nepotism that's strictly forbidden.
But back in those days that that was viewed as a positive thing.
And so I think because of that,
they, they, they took me on as an apprentice because I was a
mathematician. They weren't exactly sure what
to do with me. So, so I went through a lot of
the standard training with, with the engineers then, whereas
the engineers had to be moved around different parts of the
company to satisfy, you know, requirements for chartered
engineer status. You know, later on, you know, I
had, I had more flexibility as, as to what I did.
And so I got into various assignments which involved
programming in in various forms.
Oh wow, that's interesting. So this is before you did your
undergraduate? Yes, this is the period after
school. Before undergraduate and then
each summer while I was at Cambridge, each summer I went
back to Rolls-Royce for, for another two months.
So it was after my second year. So that would be 1980 that I
joined what was essentially the CFD group, you know, only a few
months after it was first created, you know, so it was
called the Theoretical Sciences Group.
And that that summer I was doing 2D grid generation using
conformal mapping. So directly using, using my, my,
my coursework in complex variable theory.
That was actually a paper, I think it was written by Bob Ni
at Pratt and Whitney. I don't know whether you
recognise that, that name, but Bob's a really, you know, senior
figure of that of that era. Oh.
Wow. OK, so you did.
So you're going between, but in the during your undergraduate,
did you already therefore get a sense that you wanted to go down
the more CFD route because you were doing in Rolls-Royce?
I imagine because you were doing maths you didn't touch.
Well, I guess would you do fluid dynamics in?
I guess you would do numerical methods, but would would fluids
come up? So.
So we did lots of theoretical fluid dynamics.
So I can remember thinking, you know, on on the one hand here,
here I am taking a course learning about inviscid
incompressible 2D flow over a cylinder.
And on the other hand, here I am at Rolls-Royce looking at these
phenomenally complex, you know, engineering devices.
You're clearly paper and pencil cannot take you very far.
So I was really sold on the idea of numerical simulation at a
very early age. You know, a very, very early
stage. I mean, already by the time I
went to university I'd done some amount of programming.
So I got into programming pretty early in, in, in part through,
through my mother. I mean she, she was IT support
at University of Stirling. So in the holidays I actually
did a little bit of programming on one of the big academic
systems down in Manchester. Where was that in the big
building? When was that in Manchester?
I mean, I, I didn't go there. I mean, this was remote access
from Stirling. Yeah.
So this was in, in, in the sense, you know, the equivalent
of what's now the Edinburgh Parallel Computing Centre.
Back in those days, it was the Manchester centre.
I don't remember what I did on it.
Nothing very significant. But also while at Rolls-Royce
for those eight months before going to Cambridge one one day
a week, they sent us along to what was then called Derby
Tech to sort of keep our, our brains ticking over doing
various classes, including programming.
And so I, I, I worked in an IBM system there, I, I wrote a code
to do project critical path analysis, which was great fun.
So so I've always enjoyed programming.
What? What languages would it be?
I'm sorry if I'm asking a stupid question, I'm trying.
To. Get back to like.
So I think that must have been Fortran.
It was punch cards. So that was my experience with,
with, with punch cards that that time at Derby Tech, it was,
it was a cast off the IBM machine from Rolls-Royce.
They, they, they donated it to Derby Tech.
That's, that's my recollection anyway.
So yes, I think that must have been Fortran in, in Cambridge,
we had the whole teaching lab of desktop machines and that was
basic, I think that we used. So there was a, there was a
numerical projects course in the third year, which was very good,
you know, and so that, that really sort of solidified my,
my, my interest in, in, in computational methods.
And then what about so I, I, I have to ask you one, one
question. When I was doing a bit of
research, I hadn't come across this term before, but it's quite
an esteemed Senior Wrangler. Am I pronouncing it?
Correctly. I guess you've looked at the
Wikipedia. And then I was like, then I
started to look down and I read fantastic stories about people
being paraded around. I don't know if that was the
time. So this was the top
undergraduate of maths, is that correct?
Which which you you've got. Yeah, there there was no fanfare
in my case. So yeah, I was just told
afterwards by by my tutor, yeah. OK, well, it's it's still it
shows, I guess your your ability.
Were you, did you enjoy the, the sort of Cambridge life, the
collegiate life? Was, was that something that
you, I mean, now obviously you know, you're at the other place,
but was that something that you made a, a strong impression
on you and, and sort of motivated you later on to, to
ultimately stay in or, or go back to academia?
I mean, I, I enjoyed my time at Cambridge.
I spent a lot of time doing orienteering while I was there.
So orienteering was something I
did as a child, you know, from the age of about 12, you know,
for. Yeah, for, for those listening
to this who don't know about orienteering, it's effectively
cross country running, using a map to guide yourself through
forests and over hillsides and stuff.
So, yeah, a lot of my time at Cambridge was spent going off
orienteering at weekends. I would say that I was as a
diligent student, but I wasn't particularly hard working.
Let's let's let's put it that way.
You know, I seem to remember my, my maths as being sort of a, a
nine to five activity five days a week.
And then, you know, weekends I was away.
I didn't really start working until I went to MIT.
OK, that, that that sort of sums up the difference between MIT
and Cambridge also, you know, grad student life and undergrad
life, I guess. Yeah, Yeah.
So you said the beginning that you'd, you weren't sure.
You, you were debating around theoretical physics, maths I
presume. Then as you went towards the end
of your undergrad and particularly with the summers in
Rolls-Royce, you'd, you'd put aside the theoretical physics
and you were more moving towards, I guess the engineering
or applied mathematics. Would that be fair?
I mean, I think by the time I finished that initial 8 months
at Rolls-Royce, I think probably the physics ideas had to a
large extent dropped out. Although I think equally I knew
I didn't want to work in industry.
So yeah. So maybe things were still
somewhat open. Yes, I I do remember, I'm trying
to think whether it would be end of my second year or early in
my third year. There was a talk in college by
an academic researcher just talking about the joy of being
an academic researcher and that that did strike me.
So, so that's one of those moments that I think, you know,
confirmed me in going as an academic path or, or at least
taking it further and doing, doing a PhD.
But you know, the move to MIT, my, my tutor told me about the,
the Kennedy scholarship scheme and encouraged me to apply, you
know, and so, you know, initially I went to MIT on, on
this one year scholarship thinking it would be a chance to
see the world. And then I would think about
what to do next. Having gone out to MIT, my, my
supervisor there found funding for my second year to, to finish
up the masters. And in doing that, I also came
up with a, what turned out to be a good idea for a PhD project.
And then, then he got more, more research funding for, for me
actually from US Air Force to, to carry on and do do the PhD.
So it wasn't. It wasn't the master plan by any
means, but yeah, yeah, I went out initially for one year and
ended up staying for 11. So what was the Kennedy
scholarship like? Because I spoke to somebody
else, Anthony. Oh, sorry, I'm confusing.
Oh, got a brain fog. Now the have to rise.
Now I'm just interested because I wonder if you'd actually come
across. Probably you haven't.
My my memory for names is terrible, but there wasn't
Anthony about my time. Yeah, that's why I'm.
Just. Wondering if you've in In
general, most of the Kennedy scholars were at Harvard.
There were very few at MIT and, and, and these days it's really
quite the exception to have anybody at MIT, which is a bit
of a shame I think. Yeah, Tony Purnell.
Did you ever come across Tony? Yeah, that that that name does
sound familiar. So what?
What? What's he doing now?
Yeah, So Tony, who I actually also interviewed, he's a really
great guy. So he he won the Kennedy
scholarship, went to MIT and I think it was in 80, mid 80s, so
similar time. And then he went to he then went
to work. He created his own company, but
he ultimately ended up in Formula One running the what is
now the Red Bull team. But he's now a professor at
Cambridge. Does all the aerodynamics and
CFD. I wonder whether?
Yeah, so, yeah. So I was a Kennedy Scholar 81 to
82. Yeah, and the name does sound
familiar. Basically, we we didn't really
hang out with each other. I didn't hang out with the other
Kennedy scholars. So yes, yes.
I didn't really have have have those connections, but it was a
part time experience. Oh, oh, yes, yeah, yeah.
So, so, you know, it was Kennedy scholarship that took me over to
MIT. And as it happened, and this
this was really sort of coincidence, my supervisor at
MIT had Rolls-Royce funding. This was in the days when
Rolls-Royce was starting to sell engines to the US Marine Corps
for the Harrier, and they wanted to be viewed as more of an
international company. And so as part of that, I think
almost out of their marketing budget, maybe there was a whole
pile of research funding to be spent to MIT.
And and so some of this was was going to my supervisor, a guy
by the name of Tilt Thompkins. So although my going to MIT and,
you know, landing up with Tilt as a supervisor was completely
independent of Rolls-Royce, there was still that sort of
accidental background connection.
So then my, my, my, my graduate history at, at MIT is a bit
curious. So I did did the masters in 18
months, which is a bit faster than normal, but not
exceptional. I then did did my PhD in 2 1/2
years, which is highly unusual for MIT.
So, So what happened there was I was basically two years into my
PhD and you know, it had gone very well.
I mean, this was, you know, the project that me and Mark Drela
did, you know, you know, the ISES code, you know, 2D airfoil
design code. So the project was going ahead
very well, but I got called into the head of department's office
about two years into my PhD and told that my supervisor hadn't
got tenure and would be leaving in six months time and would I
like his job. Wow.
So so I then had to finish up really quickly that that I
wasn't. Expecting that.
Wow. OK, so.
Yeah, yes, that, that, that, that last six months was, yes,
exhausting. So then, so your PhD.
What was the? What was the end title of your
thesis then for the? Something like two-dimensional
transonic aerodynamic design method.
So did you work with or be inspired, I guess by, by
Professor Jameson? Was there, was there any sort of
link into? I just, I always find it
interesting when there's sort of Brits going over to the US.
Yeah, yeah, they go and work on things.
But it was independent. There was no he didn't have a.
Connection. No, there was no connection at
all there, although he, he, he was aware of me.
He was aware of my master's thesis and actually told Tilt,
like, you know, something along the lines of, you know, they
should have given me a PhD for it, which it was flattering.
But but no, my, my, my master's thesis was using some
mathematics WKB analysis to understand some numerical wave
propagation on grids. And what it shows you is that if
you have something like the convection equation, if you have
a poorly resolved wave, it can actually travel in the wrong
direction and you can get these weird wave track wave trapping
phenomenon. Anyway, it, it, it, it, it was
something that intrigued Antony.
And so, so he was aware of me already at at that point.
I mean, I was obviously aware of him, but the, you know, the, the
stream tube idea that I had, which was the basis of the ISES
code, that was completely different to anything, you know,
in, in, in CFD at that time, you know, worked beautifully in
2D. It had had no natural 3D
extension, but, but for, for 2D wing design it, it, it was
really ideal. And so Mark, Mark Drela and I
teamed up on that. And in fact, for a while we were
going to do a joint thesis. We're actually going to write it
up as a single document between the two of us.
And our thesis committee was perfectly happy with this.
And since our thesis committee included the head of department,
I assumed that this was all fine.
And then later on in the process, the central university
somehow, you know, learnt about this and said there is no
precedent for this, there shall be no precedent.
You know, you're not allowed. And so very late in the process,
basically, you know, Mark and I had to sort of carve our work
up into two separate pieces so that we could write up two
separate documents. You know, so in, in, in the end,
I, I finished up early because I had to take over Tilt's
position. Mark, Mark finished up a year
later then, you know, he did, he got an academic position as
well, you know, so we were both hired.
I mean, there was never any question of one of us
freeloading off the other one. I mean, our, our thesis
committee were perfectly content on that point.
So yes, so it's a curious situation, but it meant that I,
I took over Tilt's office, a software engineer, almost all of
his students and all of his research contracts, including
the Rolls-Royce research contract.
So, so at that point I was then back into the Rolls-Royce
family. And how old were you then?
You couldn't have been that old. 25.
OK, that's quite, that's quite young then.
In in the US system and particularly at MIT, they they
do hire a lot of people straight from PhD into assistant
professor positions because given the tenure system if they
decide that they made a mistake, they just flush you out
after seven years. Whereas whereas in, in the
British system, you know, we, we, we, we like to see people
get a good bit of experience under their belt, you know,
before we'll, we'll, we'll hire them in, in Oxford.
Yeah. Very occasionally we'll we'll
take people straight from PhD, but it's very, very rare.
Wow. So you're 25, you're an
assistant professor at MIT, you've got a PhD, all that, and
and now you're taking over the Rolls-Royce.
So what what? How do things progress from from
there? So looking back, actually the,
the, the first thing is it took me about six months, I think, to
recover from burnout from having finished up the PhD so
quickly. But, you know, there were, there
were a certain number of plans that were already in place that
I sort of carried on supervising students.
But I guess during that first six months, I was thinking
about, you know, what, what was the first new thing I wanted
to, to, to do with Rolls-Royce funding.
And you know, I want, I wanted to do something different.
I, I can't remember how much I talked to them to understand
what their concerns were at the time.
So that what, what, what they needed.
But what I decided to do was to develop a 2D unsteady CFD code.
I think that they had had some engineering challenges in I
think it was a military engine where in military engines
there's a smaller gap between the stators and rotors.
So they're they're they're more closely coupled stages.
And as that as such, that means you get a larger level of
unsteady forcing on on on the blades.
And I think they, that there had possibly been some engineering
project where they had major difficulties with that and they
really needed tools to analyse that.
I mean, this, this was still, you know, fairly early days for,
for, for CFD, you know, so, so I think at that point, you know,
people like Bill Dawes and John Denton had developed steady 2D
CFD codes, possibly even 3D, but but not anything unsteady.
So, you know, so I, I, you know, my first code for Rolls-Royce
was one called UNSFLO, which was 2D unsteady.
Initially it was wake rotor interaction.
So you were passing in the wakes through upstream boundary
conditions and then going into doing stator rotor interaction.
So you've actually got the moving blade rows, you know,
moving relative to each other that that was initially
inviscid. I later made it viscous.
I can't remember now what I did for a turbulence model.
It's probably an algebraic turbulence model in those in
those days. Mid 80s, Yeah, yeah, I guess,
yeah. So the the the sort of unique
thing about UNSFLO was this thing called the time inclined
plane. So one of the difficulties in in
doing unsteady analysis in turbo machinery is the number of
rotor blades is different to the number of stator blades.
So you want to do a simulation that just has one blade passage,
but if you do it the natural way, that doesn't work.
You know, you don't have the right periodicity to, to to do
that. So the time inclined plane
involved, you know, usually you know, when you're at time level
N, you know, all the grid points are at the same physical time.
The time inclined plane, you know that, that, that that time
was inclined. So that, and you could incline
it in such a way that you then set up the right periodicity
condition to cope with this arbitrary blade count.
So yeah, that that, that that was kind of the the unique
point. And then, you know, there's also
some maths I did on non reflecting boundary conditions
that when you're doing these unsteady interactions with the
boundaries very close, you want the outgoing waves to go out to
not be artificially reflected from the boundary.
And so there's a whole piece of research on on, on that.
Stuff you take for granted now in a commercial solver.
Yeah, Yeah. So I mean really I was one of
the first people doing that in, in the context of compressible
flow CFD yes. I mean these, these these days
you, you've got things like absorbing boundary methods,
which is probably what you would use if you're doing far field
acoustics and electromagnetics.
But that actually wouldn't work well in in this context of
closely coupled stages with very close in boundaries.
And were you always hands on? Were you, you know, at that
time, were you always that sort of person who was programming it
yourself? You had students, but you were
still very much yes and and sort of a hands on programmer.
Yes, that was hands on. I mean generally the students
were writing their own other codes.
So, so with UNSFLO there was this software engineer Bob
Haimes. So I don't know whether you
recognise the name Bob Haimes. He, he he was responsible for
developing our visualisation software.
So VISUAL2, VISUAL3. Yeah.
Name names from the past. Bob's Bob's still at, at MIT, I
think he still hasn't retired. You know, he must be about 10
years older than me, something like that.
So, yeah. So I did most of the UNSFLO
development, but Bob will have helped me with, with, with bits
of that as well as doing, you know, he did, he certainly did
all, all of the visualisation because if you're doing a, you
know, 2D calculation, you know, you want to have some nice, nice
visualisation. It's true then, true now, isn't
it? So this was, this was in, in the
early days of Silicon Graphics and there was a company called
Stellar which was based just outside, well, in the Boston
suburb. And I, I sort of got involved
with, with them. And so we, we, we had a couple
of their, their machines, beautiful machines, you know,
multiple cores I recall as well, which was unusual.
I mean, it was a very early days of parallel computing.
And back then were you, you know, as much as you were
developing the code for accuracy and the physics side, did you,
did you still have a strong interest in the sort of high
performance computing side? Were you were you always excited
to try out different machines or have access to the machines
or did that come later? No, I was always interested.
So I mean, even even while I was doing, you know, masters and
PhDs. So I think, oh, I'm trying to
remember. So it was so probably soon after
my master's, I spent the summer at NASA Langley at at at
ICASE and down there I think I was doing programming on oh
gosh, what, what what would it be?
My mind's gone blank. There was there was Cray and
then there was the other company.
What was the other one called? So it was, it was before, you
know, the ETA 10. Oh, Cyber 205.
Yeah, that sounds, sounds maybe right.
So I guess coming out to CDC maybe so.
So I I had my first experience of supercomputing at at ICASE.
I don't think I did a lot there.
Who was there at that time? Because I've heard other people
mention about this ICASE they don't do it anymore, I don't
think, but it was this wasn't it.
So the person who led it, I think when I was there was Milt
Rose, but I think he may have retired not long after I was
there. And then for many years it was
led by somebody whose surname is Hussaini.
I'm trying to remember what his first name is.
Possibly Yousuff. No, I'm not sure.
Yes, yes, ICASE. I mean, you know, there are lots
of academics there. Who else do I remember?
Eli Turkel. I remember he was, he was there,
I think, you know, the summer that I was there.
And then, you know, we sometimes saw the people in, in the CFD
group there and oh, there was a, there was a great, great person
who headed up the CFD group. Oh, gosh, my, my, my memory
today is, is is poor. Yeah.
So, yeah. So, yeah.
So I had, I had experience with this, I think it was a Cyber 205
at Langley. At MIT, I did a little bit of
work on on the Thinking Machines CM-5, you know.
That I remember at the time, you know, there was a lot of
discussion about, you know, this is the future of massively
parallel computing, you know so I think it was 64,000
processors, but each of the processors was incredibly
elementary. And then it got blown away by
RISC computing, you know, and, and people putting together PC
clusters, you know, so I think DARPA basically bankrolled it
for maybe five years, eight years.
But then, yeah, it wasn't capable of of sustaining itself.
And then and then, you know,
Cray, Cray really got established, you know, And so
Cray, Cray was the winner. Yeah.
I so Rolls-Royce at one stage had the Cray.
So I don't remember the time scales.
So I don't remember whether UNSFLO was ever run on the Cray
or not. I think it's possible it was.
And were you travelling back when you were MIT working for
Rolls-Royce, Did you come back to the UK, to Derby to sort of
have meetings with it or was it slightly sort of separated?
I mean, I probably came back twice a year at most.
So. And, you know, to some extent I
would be coming home to come home and see people.
And then while I was here, I would visit Rolls-Royce.
And to some extent, you know, a trip might, might be motivated,
you know, more more primarily because of Rolls-Royce, you
know. So, yeah, I guess probably twice
a year was the norm in those days.
So how did things progress then at MIT?
So you were working on the Rolls-Royce projects that, Yeah.
How did, how did, how did things evolve during that time?
I mean, as well as working on the Rolls-Royce projects, I had
some amount of funding from US sources.
Not a lot, but things I think. Office of Naval Research Air
Force Oh, we, we, we bought the stellar machines with a grant
from DARPA. So again, DARPA was really
active in funding new technologies to see whether, you
know, these really were useful or not.
That that is probably the worst proposal I have ever written.
But but, but my excuse is by the time I submitted the proposal,
my temperature was 102 or 103. I was going down with glandular
fever. Oh, I've had that.
That's bad glandular fever, isn't it?
Yeah. So in in, in the way I was lucky
mine was sufficiently bad that I had to be admitted into
hospital. And so they then pumped, pumped
me full of, you know, various antibiotics and stuff.
And so I recovered well. I was ill for like 6 weeks or
something, it was horrible. Yeah, I was.
I was probably in hospital for a week or two and then I was
sent home and told to stay at home and and recuperate for
like 2 months or something. And and So what I did was to
write up a 60 page document documenting all of UNSFLO so
that that that was the best documented code I ever wrote it.
It's always this, this rule that people hate writing detailed
documentation and, you know, just just laying out all the
details of the numerics, all of the things that your future
people modifying the code need to do.
You know, it's very hard getting getting students to do that.
So anyway, I yeah, yes, I'm trying to trying to remember
exactly when, when all these different things happened.
Yeah. So.
So you're progressing at MIT, you're an assistant professor.
Did you feel that you would always stay there or what
started to get into your head about coming back?
To so I, I, I had a good, good job.
Obviously they're, they're at MIT.
I was part of the gas turbine lab and there were wonderful
experimentalists there. And so, you know, we, we, we did
lots of good works of comparing numerics with experiment.
And there's one paper we, we, we have an unsteady heat transfer
where in the sense neither the experimentalists nor me had
great faith in our own research and yet the results matched
wonderfully. We were delighted.
I, I did feel to some extent, not quite a fish out the water,
but really I am at heart a mathematician.
You know, you, you asked me at, at the beginning, you know, was
I the kind of kid that tinkered with devices?
No, I wasn't, you know, so I, I, I really wasn't by nature an
engineer in that sense, by by nature, I'm an applied
mathematician who enjoys mathematics and enjoys seeing it
being useful in the real world.
I'm not fundamentally at heart an engineer.
And so in that sense, I did feel a little bit constrained being
in the aeronautics and astronautics department, which
is, you know, where I was at MIT.
And that that I was sort of most aware of when doing things like
the non-reflecting boundary condition theory, because that
was very much maths theory, but publishing it in in engineering
CFD, So journals, you know, AIAA Journal.
So that yeah, I think also, you know, getting married and
thinking about where, where you want to have a family and raise
kids and things like that. You know, that that that was
part of it. And then the other part of it.
And I forget the exact sequence of all of this was on one of my
trips to Rolls-Royce, I was called in to see the chief
engineer who said, oh, by the way, please let us know whenever
you want to come back to this country and we'll sort it.
So I had that standing offer from Rolls-Royce that they would
organise it for me or that I could basically choose where I
came back to. So yeah, so I decided, yes, I
wanted to, to, to come home and, and, and I chose Oxford rather
than Cambridge as the place to come to partly.
Well, you see in, in Cambridge I'd have probably been in
engineering, not in maths. You know, Cambridge maths was
always kind of anti numerical analysis.
There was certainly a chunk of the faculty who felt that you
know the computer is what you used if you weren't clever
enough to do it properly. Old school.
Old school, yes, whereas Oxford really had had embraced
numerical methods and had very, very strong group here and in
those days the numerical analysis group, although they
were a group of mathematicians, they were in the computer
science department. And so there were also people in
computer science on the parallel computing side.
And so that that was an attraction to me as well, you
know, to have them as sort of neighbors.
As it turned out that that didn't work out because soon
after arriving in 92, it was the first dot-com boom and all, all
of the parallel computing people left to set up companies.
So anyway, but but that was part
of the motivation of choosing Oxford.
Ah, OK. So yeah.
So, you know, Rolls-Royce kind of organized it.
I mean it, it had to be a properly advertised and competed
for a position. So I was in this strange
position of helping to write the job description, possibly even
the advert for a position that I then applied for.
You know, there are various parts of my career which I, I, I
look back on. And now, now I realise just how
peculiar they were at the time.
That's right. So you technically came into the
computer science department? Yeah.
So in those days it was called the Computing Laboratory.
Yeah. And that was a Keble Road always
OK. And so that was like a
Rolls-Royce. So it was a Rolls-Royce
readership in CFD and then and then they funded me to set up a
whole research group. So I, I have a lot of funding
from them for a, you know, prolonged period.
So really from 92 through to 2008 is when I moved to maths,
you know, so, so, so for those sort of 15 years, you know, I, I
had a lot of Rolls-Royce funding.
So this is the bit that I was wondering about.
So you came into computer science for CFD and is this the
time then when I guess what most people recognize as the
Hydra code is, is this where it begins or did it actually
begin even when you were MIT? When, when would when would you
say was the start of that? Journey.
So I would say that Hydra proper started in about 96, but there
was there were various bits of research that in the sense laid
the foundations for Hydra. So while I was still at MIT, you
know, so I was thinking about what to do after UNSFLO.
And so my plan after UNSFLO was that we really wanted something
that would be a design tool for doing complete engines, you
know, so no no longer single stage, really looking
multistage. I was also interested in the
idea that the unsteadiness could be done from a linear
perturbation point of view rather than doing nonlinear
unsteady. So so also linearise the
unsteady equations, look at, you know, harmonics.
And so doing this for both flutter and forced response.
The key technical issue there is whether it was legitimate to do
linearised harmonic analysis of shock capturing.
And so I had a student who did a project at MIT to prove that,
show that that was was a legitimate thing to do that as
long as you so slightly smeared the shock over a few grid points
that that the linearised analysis did do the right thing
and did, yeah, you know, you could do it on that basis.
So that was the precursor work at MIT.
And then early on in Oxford, I don't think I'd started this
coding before I moved. I did a code called
SLIQ—S-L-I-Q. So steady, linear and quadratic.
So the idea was you did you know the non linear steady state,
you did a linear perturbation analysis for the unsteady
effects and then the quadratic was to it's kind of a formal
asymptotic expansion to get the mean flow changes due to the
second order quadratic effects.
So, yeah, so I developed SLIQ early in, in in my Oxford days
and and you know, I had a student who who, who worked on
that with with me. So that was one piece of work.
Another piece of work was back in the 1992–93 era, there was
funding. I think this came from, you
know, the UK Department of Trade and Industry DTI.
In those days there was an initiative of setting up
parallel application centres in various parts of the country.
And so I, I had a colleague on the computer science side in
Oxford, Bill McColl, who'd applied for that funding even
before I arrived. And so had funding for an IBM
machine, something called an SP2 and and also there was funding
there for, for research and also also match funding.
I think so. So I had 50:50 matching from
Rolls-Royce and so did the project there on developing.
So in the sense of support layer for doing distributed memory
parallel computing. So this was something called,
you know, we called OPlus Oxford parallel library for
unstructured solvers. Although although orally OPlus
doesn't doesn't sound right, but written down it looks good.
So, yeah, so, so I think Bill, yeah, must have been in Oxford
after I arrived from maybe two or three years before he left in
that.com boom. He was one of the people I hoped
to work, worked with a bit more on the parallel computing
side. So, so we got a whole pile of
funding half and Rolls-Royce half from DTI to develop this
parallel application framework.
There was another code that at the postdoc Paul Crumpton, who
actually did most of the development of that software and
wrote the CFD code as basically a test bed to check
everything worked correctly. But that was never intended as
a CFD code for Rolls-Royce. So then that's what led into
Hydra. So basically there were the the
ideas of doing steady post linear perturbation tested out
in SLIQ and there was the parallel framework in OPlus
that we'd developed thoroughly tested out.
And so Hydra then what was built on those foundations now.
So Hydra it dropped the idea of doing the quadratic piece.
So, so it's non linear steady, or at least in its original
incarnation, non linear, steady linear perturbation for flutter
forced response and then adjoints of all of those for for
the design optimization, both steady and and the unsteady
aspects, you know, so the adjoints was completely
motivated by the work that Antony Jameson was doing, you
know, on on, you know, the aircraft side, except that I
chose a different technical approach in as much as Anthony
always viewed the adjoint being developed at the PDE level, you
know, formulating the adjoint PDE and then thinking about how
to discretize it. Whereas I followed the the
so-called discrete adjoint approach, where where you take
the non linear discrete equations, you linearize those
sort of element by element and then you take the transpose of
the matrix to define the the discrete adjoint.
You know and. Yes.
How did you, just out of interest at this time, maybe
also MIT, but Oxford, how active were you in, you know, the AIAA,
the the sort of turbo machinery conferences were you, were you
sort of always, were you at these events and saw these
people or was this more of a like direct industrial
engagement? I'm always interested, like now
I go to the AIAA conference. I'm always, just always
wondering what it was like, you know, before and was that still
the main venue I guess to go. Yes, yes, that, that, that, that
was the main venue. I, I sometimes went to the
ASME IGTI conference, but I went more, more to the AIAA
conferences. You know, I think there was more
discussion of CFD at at AIAA and in, in, in particular, you
know, the AIAA CFD conference. I mean that that was really my
home, you know, sort of during this period.
I would say that, you know, the ASME was more on the
application side. Yeah.
So I, I, I, I can remember going to two or three of those, but
it was the AIAA ones which really were.
Where was was this Reno? No.
Where was the CFD? So Reno was the January
conference, the CFD conference, what was in the summer.
So I I remember well, I remember one in Snowmass in, in
Colorado. Yeah.
Beautiful venue. Yeah.
Good, good. Orienteering, right?
Well, yeah, yeah, except it's high, high enough up that you
really wouldn't be wanting to run at that altitude.
No, Yeah, yeah. I mean, this is this is a ski
resort that in the in the summer.
You know, they, you know, are used for conferences because
there aren't that many people who want to go hiking.
But I remember Snowmass. I remember Hawaii.
I remember there was 1 in LA. Yeah.
I mean, it just hopped, hopped all over the place.
Yeah. So.
Yeah. So I was really mainly in those
days going to engineering conferences, not so many, I
guess some maths conferences, but, but in those days, yeah,
most of my publishing was in in engineering journal still at
that point. But did you struggle?
I'm interested because I, because you have such a strong
maths background, did you ever struggle getting accepted or
being in that blur of what journal, what conference is the
right place for maths? And then I always find this
interesting that, you know, it's something too applied or too
fundamental. Did you find a sweet spot or was
there still a little bit of a straight frustration that your
deep maths wasn't understood by everybody?
Do do you know what I'm getting at or was it not an issue then?
I mean, I, I think I was
probably in those days still viewed more as an engineer than
than the mathematics, you know.
I mean, you know, if you have a PhD in in aeronautics from MIT,
you know, you're, you're an engineer.
Yeah. So, I mean, there was never a
question of the engineering community not accepting me.
You know, I think, you know, my, my evolution has, has been one
of the of the maths community accepting me.
OK, OK. Yeah.
So, yeah. And that, that, that I guess,
you know, happened more once once I moved over into the Maths
Institute. But yes, I guess I felt, I mean,
having made the, the move to the numerical analysis group in
Oxford, then in a sense I felt I was back amongst
mathematicians. But but I was still very much at
the engineering end of of of the group.
You know, I think I was probably 40 before I wrote my first
paper that had a theorem and a proof in it, you know, So I
mean that that's kind of needed to be a mathematician.
So what was the link to the engineering department at that
time like because there's was, was it the case that that was
more experimental work and the sort of CFD was mainly computer
science? Is, is that sort of, I always
found that unique in Oxford that, you know, like I said, MIT
engineering where like, yeah, I guess Oxford's a bit different.
So in in Oxford at that time we had three UTCs.
So the UTCs are the university technology centres that, that
Rolls-Royce set up. And so I had mine in CFD, there
was an experimental one in heat transfer in, in, in engineering
science. And then there was one in
materials which I didn't didn't have any interaction with
except one of the profs. There used to be an orienteer
back in my case. So there was.
A UTC just for CFD. Yes, yes, yes.
So this was my, my own little UTC.
I was the only academic in it. So and, and possibly within the
Rolls-Royce family, that's slightly unusual to have a UTC
that only has one academic. Generally they're they're bigger
than that. But it was a funding mechanism,
I guess, and you were doing the work.
Yes, the funding mechanism it it, it involved me in all the
UTC directors meetings. I mean, you were part of the
family. And I mean that that's kind of
important in the sense that Rolls-Royce really knew how to
work well with academics, that you were part of the family in
the sense that you knew all the problems as well as, you know,
the, the achievements of, of, of Rolls-Royce.
You know, they, they, they, you know, they didn't hide anything
from you so that you could think about what you might
potentially do to, to, to help them address some of their
challenges and things. So, you know, it really was a
very good collaborative experience when, when you're
working with industry, you know, it's important that both sides
realise that what the other wants out of the relationship
is, is different, you know, and, and, and so you're always
looking for this sort of win-win arrangement, you know, the,
you know, so they understood that, you know, for us, it was
important for the students to publish papers, to write
dissertations, you know, that there would be times when they
would be utterly focused on writing their dissertation and
not doing any more research, you know, but equally, I, I
understood what Rolls-Royce needed out of the relationship,
which was primarily software, but occasionally if, if there
was a particular engineering thing to be investigated, you
know, we may occasionally do some, no special calculations
just for them, as it were, rather than as, as part of the
research. Yeah, I always found that
interesting that some it takes a special company to understand
the value of academic engagement and have the patience and the
long term vision that it's not just cheap labour.
Yes, you know that requires you.
You have to be very careful to make sure it's never just cheap
labour. I think Rolls-Royce was possibly
slightly disappointed that they never ended up being able
to hire any of my students. You know, that's, that's, that's
the other thing that, you know, Rolls-Royce would ideally
like from a UTC is as a source of, you know, people to be
employed, you know, and that that never quite happened.
But, yeah. But certainly they, they, they,
they got their money's worth in terms of CFD codes.
So, yeah, and I guess like in today's world probably, you
know, you'd create a start up or something, write a code, you
know, I guess at that time there was a more traditional link to
the to the company, right? You know, in terms of.
Yeah. I mean in, in, in this area, it
would be tough to do a start up.
I mean, I guess Juan Alonso's done, done the startup.
I haven't talked to him recently as to how, how that's going.
I mean it, yeah. At, at, at one point in parallel
computing, I, you know, once I moved out of CFD into into
mathematical finance, I, I tried setting up a spin off.
No, it's just more hard work. Yeah, I'm, I'm, I'm, I'm
fundamentally an academic, not, not a start up guy.
The reason I say it is because, you know, speaking now to
yourself or, or to Antony, and I guess that like now CFD has
become very dominated by these huge multibillion dollar
commercial companies. I guess in the 80s, that was
before the time, wasn't it? It was before the, the Fluent
and, and, and the, the OpenFOAM and it was still, you wrote your
own code. I assume that's old.
Was that one of the reasons for the Hydra development that they
couldn't just buy something off the shelf that wasn't a
company that could just sell them a capability they felt they
needed? Yes.
And even today I don't think there's a company that could
sell them what they need because the turbo machinery
requirements are really very specific.
So I mean I've I've not kept up with the discipline.
So I don't know what ANSYS Fluent has as a capability these
days, but you know, the ability, for example, to have
flutter calculations being performed on a single blade
passage with an inter blade phase angle between the
passages. You know, that's such a unique
requirement of turbo machinery that I'm not sure ANSYS views
the market as being big enough to develop that capability.
And then, and then you get into things like real gas effects,
you know, so, so, so Hydra doesn't assume a fixed gamma.
You know, it, it, it has a general, you know, energy
temperature relationship in there.
So again, I mean, I guess things like that ANSYS could add in,
but they would be additions for particular customers and, and
they would charge accordingly. I mean, one of the reasons I got
out of the CFD business is it's not clear to me long term, you
know, what Rolls-Royce will do for their next CFD code.
So I mean, Hydra's just celebrated its 25th anniversary
at, at, at Rolls-Royce. I would say it'll remain the
corporate, you know, primary code for at least another 10
years because it takes a long time to, to introduce a brand
new code. And you know, they, you know,
they're doing work with Spencer Sherwin on his Nektar++ code
with the thought that they will use that in, in applications
where they want to really do DNS or or at least high resolution
LES, but, but that won't be part of their sort of standard, you
know, design process. So I think it's, it's harder to
see now. Yeah, companies funding a brand
new code development. That's kind of what I was
hinting at, that it does seem as if that that generation in the
80s and the 90s throughout aerospace was a time of real
innovation and development. It was.
Themselves. Yeah, I mean when, when, when I,
you know, joined Rolls-Royce's first CFD group in 1980.
You know, most of the design methods were one-dimensional
design methods, you know, with sort of mean, mean line, sort
of, you know, I mean really mathematical models rather than
numerics. So, yeah, I mean it, it, it's
been wonderful just being part of that whole process of, you
know, you know, the development of computational engineering
and, and seeing it go all the way from 1D, 2D and 3D, from
inviscid to viscous steady to unsteady and then the adjoints
and everything. You know, it is interesting.
So looking back over your lifetime and looking at the
progress in computational engineering and then of course
the progress in parallel computing and the high
performance computing, I mean. Both of them, yeah.
It's really stunning looking back over the years.
Yeah, that's what I was wondering about is so you, so
you came into Oxford doing the computer science.
I was just looking at the and I guess this is where we have a
slight shared connection into some of the people.
How did and you mentioned also about the government setting up
some, you know, parallel computing.
I guess how did that evolve the HPC angle, Oxford, maybe some
of the e-Research ideas, you moved into maths.
When did how did that time? And I guess that probably aligns
to also when you had a declining interest in CFD and more into
the computing and other areas. Is that fair to say?
Were they aligned a little bit or were they sort of separate?
OK, so yeah, yeah, let's let let's.
Try to. I've asked you like 3 questions
in one. Yeah, yeah, yeah.
So get get the timeline straight in my mind.
OK. So the main Hydra research
period was sort of 96 to maybe 2004, 2006, something like that.
And the Hydra code was built on top of this OPlus parallel
layer that we had developed earlier as part of the DTI
funded activity. And so incidentally, both OPlus
and Hydra, you're the IPR belongs to Rolls-Royce.
So OPlus was very much developed in the time of single
core CPUs, RISC-based CPUs, you know, so, so there, you
know, there was good performance there, but single core.
And then what was happening over time was that CPUs were, were
going multicore and, and then we had GPUs that came along.
So, so again, I've, I've always been interested in the latest
computing technology. So I think it must have been
sort of late 2006, early 2007. An ex colleague, Mike Rodger, I
don't know if the name means anything to you.
He, he set up a company spin off from Warwick actually, which
sold parallel, you know, systems, you know, parallel
clusters. And he was the one who said to
me, there's this new thing called the GPU and this, this,
this new language CUDA. Actually, I'm not even sure if
he told me about CUDA. He may just have told me, you
know, there's this new hardware called GPUs.
It's really impressive. You know, you ought to have a
look at it. And at about the same time there
was a company based in Bristol called ClearSpeed, which had
people in it who I think had the background coming from the
INMOS transputer days, you know, so, so I first tried the the
ClearSpeed card and I was impressed by what it was capable
of, but I wasn't blown away by it.
And then actually I had the visiting student, a Chinese
student, I forget how, how he came to me, but he, he did the
ClearSpeed work. And then I said, well, now, now
let's try this GPU. And he came back to me with it
within like a couple of weeks with the code performing just
incredibly fast. And I was saying, OK, you've
clearly messed up the timing. And so it took took me a week to
convince myself, no, I mean it really was performing that that
well. So, so this was right at the
beginning of CUDA. So I think we, we started with
the 0.9 beta release or something like that.
And, and so I was just stunned at, you know, the power of, of
the GPU. Now this was when I was already
starting to pivot into mathematical finance.
So I was actually looking at it for doing Monte Carlo
simulations in, in finance, which is why NVIDIA then got
interested in me because they saw that as a potential market.
So, yeah, so, so I was pivoting to finance, but I was also
interested in the power of the GPUs.
So after two or three years of that, I'm trying to think
exactly on the timing. Again, I was interested in doing
an upgrade to OPlus which could then incorporate both GPUs and
multicore CPUs. But by the way, just before you
go on, did so did you if it was 2006, 2007, did you interact
with Ian Buck at all then on the CUDA stuff or was it on
people more in the UK who? You no, no.
So Massimiliano I, I interacted with Ian.
Ian, I probably met in in the same way that I met Jensen at,
you know, drinks things at GTC.
So, so I was, I think the number 2 CUDA fellow.
I think. I think the first CUDA fellow
was somebody in India. And then I was #2 like a month
later. So yeah, yeah, I mean, yeah.
So I was going along to the GTC, you know.
I have to say to the, I guess to the people is that a bit of like
self-proclaimed interest just because now the team I'm in has
those people in it. So I'm always intrigued when
they're like, oh, you, you know that person and this person
knows this person. So it's kind of interesting,
especially because you were, I mean, now everybody knows NVIDIA
and GPUs. But 2006 seven was really early
on, wasn't it? Yeah, yeah.
And, and in fact, some of the people from ClearSpeed moved to
NVIDIA in part in influenced by my feedback to them.
Yeah. They then saw the writing on the
wall and moved, moved ship accordingly.
So, but yes, I don't think I, yeah, I don't remember talking
to Ian Buck at all. Massimiliano was was was one of
my early contacts in in in the Bay Area.
Yeah. So, yeah, yes, I sort of came
back then to do this upgrade from OPlus to OP2.
So that was with Gihan Mudalige and István Reguly—you know
both of them. So that had funding from both
Rolls-Royce and, you know, the UK EPSRC, you know, government
funding. So that has in a sense, you
know, protected Hydra's future by, by up upgrading the, the,
the underlying computing harness for, for, you know,
modern systems. But but I haven't really done
CFD as such since about 2007. I think that was the last
time. So what was, yeah, maybe that's
the elephant in the room. Why what, why was the, I mean,
you sort of hinted towards it earlier, but what was the reason
to maybe move away from CFD and move into the, you know, the
financial side? Was that the maths angle moving
to the maths department sort of just wanting new challenges?
The computing side What? What was the ingredients to
that? Yeah, I'm just, I'm just
thinking what, what, what what to say.
Yeah, let's, let's let's be be open about it because I have
been been open about this, you know, in, in various settings.
I got, you know, more than burnt out developing the Hydra code.
You know that there were lots of aspects of the Hydra code that
worked very well, but they had major problems early on with
numerical stability which got me horrendously stressed out to to
the point of serious consequences.
So, you know, for about 6 months it took to, to recover from all
of that. And I decided at that point that
it's a got to a situation where in a sense, too much
perspiration and too little inspiration.
It, it, it's not being fun, you know, managing a large software
project is, is tiring, you know, and, and, you know, I'd,
yeah, I'd, I'd had enough. So, you know, I sort of finished
up the things that needed to be finished up, transferred
things to Rolls-Royce and they've, they've continued, you
know, the development subsequently.
So I decided I had to had to do something fresh different.
I had to get out of big codes. Yeah.
So in, in engineering style, I, I thought, OK, where's the
money in terms of where, where's the research funding?
What do, what do people care about health and wealth?
So, so computational biology and computational finance, those,
those were the two areas I contemplated moving into.
And computational biology, I, I know no biology.
So I'd have been starting from ground zero on that.
And I also wasn't convinced there was the research funding
in in that area, although I was proved wrong on that, you know,
you know, there is actually plenty of funding there.
Computational finance, there was a very good mathematical
finance group in the maths department but they didn't
have a lot of numerical expertise.
So, so for me that that was a natural fit.
And So what I did was I actually transferred from computer
science into maths to join the mathematical finance group.
And then a couple of years later the rest of the numerical
analysis group kind of moved, moved across behind me into
maths as well. And so now, now I'm, I'm head of
the, the numerical analysis group now, you know, so, so in a
sense in, in, in the last few years I've sort of transferred
back from the finance group into the numerical analysis group.
As which ultimately sort of came with you in the fullness of time
from the computer science into the into the.
Maths, yeah. When when when maths got its new
building, there was the opportunity for the numerical
analysis group to move and it was a take-it-or-leave-it kind
of opportunity. And you know, they, they, they,
they moved. And for people listening, it's a
lovely building. It's a very nice.
Building also, you know, when I
joined computer science in 92, the numerical analysis group was
half of the whole department, you know, Wow yeah.
So what happened over time was the computer science side grew
and the numerical analysis side didn't.
And so by by the time the numerical analysis group moved
over in 2010, yeah, they were basically pushed out
effectively, you know, you know, you know, that it didn't make
sense for them being in computer science any longer.
I mean, historically they were there because that's where the
computers were, you know, and, and that, you know, that is a
history that has happened elsewhere.
I mean, you know, numerical analysis at Stanford for a long
time was based in computer science.
And I think, again, historically, it's because
that's where the computers were. I do always find this
interesting and that's why I just wanted to, you know, get
the I get the quick history of like places like the e-Research
Centre and others because in some ways I understand the logic
that where does some of these places fit?
You know, there's numerical analysis, there's computers,
there's high performance computing, there's an
engineering application, there's a pure application was the was
was that sort of e-Research centre and I believe there were
others around the country was the initiative to try and bring
them together to in a more collaborative way Was that was
that sort of the initiative? I know that they've subsequently
largely folded into other departments now, but.
Yes, I mean, it it there was a massive funding initiative to
fund this e-Science and yeah, so, so Tony Hey was was the
person in charge of that. And I'm not sure what exactly
the intent was. How, how explicitly they wanted
it to be an interdisciplinary effort.
Maybe. Maybe they did so in Oxford.
Yeah. When, when, when they round up
the usual suspects for high performance computing.
I was, I was one of the usual suspects.
And so I was one of the four people who put in the Oxford
bid, you know, to get OERC initially, I mean, at that point
Anne Trefethen was Tony's deputy and then she later, you
know, joined Oxford and and and became head of OERC.
So I think certainly computer science in those days in Oxford
was very theoretical, especially after the parallel
computing people like Bill McColl had left during the
dot-com era to some extent in Oxford there were tensions
between engineering and computer science as to where some of
the more applied computer science activities should go,
should should go. And then OERC was, was just,
yeah, another, another location to have such things, you know,
so, so eventually it made sense for OERC to be merged into
engineering. I think at the time computer
science expressed a strong view that they they did not want to
be the destination. So.
Anyway, OK, but maybe on to then more the maths side.
I mean, I'm, I'm intrigued because maybe to explain at more
of a higher level, if you can, what what are the similarities
between some of the mathematical finances and maybe CFD?
What are the, and obviously you're known for which, you
know, I only understand the basic basics of it, but the sort
of multilevel Monte Carlo, which I as soon as I saw the
description, always like a multigrid.
It always makes me interested that there's some of these, you
see it with AI today that some of these CFD solutions to
problems are now being applied to new areas.
So what? Yeah, what are the sort of high
level things that makes the link between CFD and maths, I
guess. So in in mathematical finance,
there's basically two kinds of methodology.
You can approach it from a PDE point of view, where in in one
sense you you have a PDE that describes the evolution of the
probability density function for, you know, a stock having a
certain value at the time in the future.
And there's a corresponding sort of adjoint of that to give the
value of of various financial options or there's the Monte
Carlo approach where you simulate lots of these different
possible future trajectories of of the stock and then and
then say, OK, given that family of solutions, what's the
average pay off of your financial option?
So when I initially moved into finance, I was focused on the
PDE side because it's basically convection diffusion PDEs.
And I thought, hey, you know, this is, this is a no brainer.
I can just bring all the CFD techniques over and, and, and do
things here. What I quickly found was other
people have beaten me to it in terms of moving over from CFD.
So particularly Peter Forsyth, University of Waterloo in
Canada, who I think Peter had come out of the oil reservoir
CFD area if I remember correctly.
And so he he did a lot of the pioneering work in terms of PDE
methods, you know, numerical methods for for finance.
And so there wasn't, as it happened so much, you know, left
for, for me, as I had maybe thought.
And then I mean this, this was over a relatively short period
of time. I thought that I would probably
have to teach a course on Monte Carlo methods, you know,
because, you know, we, we, we had still have an MSc in
mathematical and computational finance.
And I, I teach numerics on that. And you know, we would need to
teach them about both sides. And so I took a 2 day course put
on by a couple of professors from Columbia University in, in,
in the US, put on in, in London for, for London finance.
People managed to, to, to
convince the, the, the department to, to pay my fees
for that. They, they, they, they gave me a
50% discount as an academic, but Even so, it was a costlier
course to attend. Anyway.
So I went to this course and they were teaching me about,
about, about Monte Carlo methods and talking about doing
sensitivity calculations. And I went up and talked to Paul
Glasserman, the lead guy in in a coffee break and said, well,
this is all fascinating, but I presume that of course you, you
actually use adjoint methods to to compute these sensitivities
more more efficiently. To which he said, what?
So yeah. So just pure, pure luck.
I was able to introduce adjoint methods to the financial Monte
Carlo community for doing sensitivity calculations.
So I did a paper with with, with Paul Glasserman in a finance,
so industry magazine really, rather than as a proper
academic journal. Because from my point of view
there was absolutely nothing new mathematically.
This was just a new application.
This is a paper that went, went, went by the title of Smoking
Adjoints, which helped helped. Its notoriety.
This is a journal or, or trade journal that liked puns in their
titles. And so I would never dare do
that in an academic journal. But anyway, so yeah.
So I so I got known for, you know, the adjoint work, which is
really sort of taken over in, in in the finance sector.
And then the multilevel Monte Carlo.
Yes, I mean, you're right to take, you know, the analogy to
multigrid. I mean, multigrid is such a
fundamental part of CFD that it was natural in getting into
Monte Carlo methods to think, well, is there anything
analogous that that that we can do here?
And so, yeah, I came up with with, you know, the multilevel
idea. And it's one of those things
that like multigrid itself, I mean, it's such a simple idea.
It really ought to have been thought of ages before.
But because I came in from a different background, you know,
this was, this was part of my toolkit, you know, it, it, it
was a fairly natural thing for, for, for me to do.
And so I've kind of been living off that and extensions of that,
you know, for the last 15 years.
And what about the computing side that the GPU side, you
know, is that is that being just because of, of, of an
interest? How much of that and forgive me
for for not knowing, but how, how much is that shaped Also on
the maths side, how, how much is that acceleration?
You know GPUs to CFD is well known but is it a similar idea
on the maths side? So, I mean, most maths research
just doesn't need lots of compute power.
I mean, these these days more and more, you know, our, our
students do things in Python And, you know, maybe some will,
will use the JAX package within Python to get performance.
But a lot of work. No, nobody worries about, you
know, performance, you know, so I'm, I'm kind of an unusual,
yeah. So I, I, I still teach my, my
CUDA course every year with, with Wes.
You know, a few weeks ago I taught in, you know, a one day
OpenMP, you know, some mini course for, for PhD students
just to introduce them to this forgetting performance.
But there's very, very few students who are particularly
interested in, in that. It is an interesting question.
You know, where, where does that kind of work belong?
You know, to what extent does it belong in computer science or
maths or engineering? You know, I guess I'm somehow a
product of, of my time and it's not clear that there will be a
new generation of people like me, at least not in maths.
Yes. I'm not sure where, where the
next generation of me sort of lives, you know, so, so there's
Wes. Wes is in, in engineering now,
you know, having moved with, with, with OERC.
So, you know, Wes is now kind of my successor within the
university. He's, he's now Mr. HPC, you
know, there will be individuals in departments like physics and
chemistry and biochemistry who, who have their expertise in
HPC, but we don't really, you know, we're not a community as
such now, I would say, and we're still struggling a bit to
organise graduate teaching across the university.
I mean, this is something where Oxford and I think Cambridge
are, are poor compared to our American counterparts, where
you'll, you'll have graduate courses offered by one
department taken by people from across the university.
Yeah. We, we don't do enough of that
Our, our, our, our CUDA course is, is unusual.
I mean, this this year, I think we're currently up to about
well over 100 people signed up, of whom 65 are Oxford people
and another 35 externals, you know, and so the Oxford people
do come from across the university, but that, that's
very unusual in, in the Oxford setup.
I'm, I'm, I'm trying to get more of that happening.
You know that we we have a much more systematic training in, in
advanced computing because you know there are needs across the
university. I mean, I have to ask the
obvious question, which is you know, you've you've pivoted once
I guess to the maths were you, were you?
And are you still tempted on the AI given that it seems your
skills are probably absolutely perfectly aligned, which is
strong maths and tech, engineering and HPC?
Yeah, So the quick answer is no. So it, it, it is curious.
I've I've got so three points of contact with with AI.
I've, I've got the, the GPUs I've got, you know, stochastic-
gradient methods are very close to the stuff I'm doing in Monte
Carlo. And there's the adjoint, you
know, so. I mean, certainly at one point
in the past, if you looked at the source code for PyTorch,
there were more references to my adjoint publications than than
any other academic, you know, So it is, it is interesting how,
how, how the adjoint stuffs got got, got picked up.
But, but no, no, it, it AI is for, for for a new generation.
No, I'm, I'm, I'm not doing another pivot.
I'm, I'm I'm happy to keep things going on on the HPC side.
So I will continue doing things
with, with, with GPUs I've, I'm, I'm interested in the
potential of FPGAs for doing finance calculations.
It it it's amusing when when I arrived in in 92 in computer
science, one of my colleagues then in in computer science told
me, you know, this is wonderful new technology Mike called
FPGAs is going to absolutely revolutionize everything that
you're doing. You know, you really need to
learn about it, and that's kind of still the message.
Yeah. I was just about to say they
haven't been right in the last 30 years.
Doesn't mean that they may not be right at some point in the
next 30 years. So.
And there's a story for GPUs from that.
It is that, you know, you were there early in the day, But I, I
guess it's Jensen always says that, you know, it's, it's taken
like 30 years to sort of get to this point where there's been
this massive, you know, use because of because of AI, but I
guess it was because of people who are early adopters it like
yourself. It's it's been a slow
progression, but suddenly it's all come together.
Hasn't. It, I mean, it has been
fascinating being sort of in, in, in the company of, of NVIDIA
during this whole evolution. You know, that, you know, I went
to all the GTCs in, in the early days and I, I remember being
there, you know, when ImageNet— you know, sorry, not
ImageNet—when AlexNet won the competition, yeah, for, you
know, image recognition, image classification.
And then the next year, the top 10 competitors won with, with
GPUs, you know, and, and you know, the way that NVIDIA
pivoted was, was impressive, but they were always on the lookout
for the killer application. You know, so right at the
beginning they thought, at least this is my impression that they
thought that computational finance might be the killer
application. And so that's why they were
interested in in what I did in implementing, you know, the
first random number generator on, on, on CUDA.
You know, I think in, I think at that point, the value
proposition wasn't sufficient to persuade the banks to take
highly paid quants to, to rewrite all of the software.
And so the adoption rate was, was, was slow.
But yeah, yeah, the company was always on the lookout for that
killer application. And, you know, when when they
saw, you know, those early days in AI, they thought, yeah, let's
let's let's double down on this.
And, you know, first, first on the software side and then
increasingly on the hardware side as well.
Yeah, I mean, it is. Fascinating.
I just find that the CFD in some ways is still a niche industry
at a global level. You know the amount of money and
the amount of people. But I do always find it
fascinating that so many things have originated in CFD.
I mean, Ian Buck's PhD was CFD, right?
CUDA, as far as I'm aware, there's like a fluid dynamics
application, you know, like with yourself with the early days
of CFD. OK, now you've it's.
It's funny how I guess because it was one of the original hard
problems to solve. Yes.
I mean CFD, I guess was a prime driver for HPC for for a long
period, you know, So yeah, I mean, both in terms of as it
were open, open CFD and all, all, all of you know, the
nuclear weapons stuff as well. Yeah, which is sort of related
very closely, you know. So whereas now you know, within
the UK the biggest computers or the most IT spend is in the
banks, you know, you know, So you know, the amount of money
that Rolls-Royce spends on compute per year, I would
imagine is, is less than any any one of the big banks in
London. I don't know that for a fact,
but no. Probably.
That's the case, you know, of sense.
Yeah. You know, so these days it it's
the money in AI which is driving the hardware
development, you know, clearly.
And so, you know, you know, CFD is no longer that driver.
I guess CFD was also driving a lot of development of numerical
methods within academia, you know, and academia tends to be
at the sort of bleeding edge of the technology.
You know, again, all of those cheap bodies, Yeah.
Bright, bright young minds and well, seeing just how much they
they can squeeze out of this, this new hardware.
But yes, what are the big drivers these days other than
AI? Yeah, Yeah.
I mean that so much is is is based on that now.
Yeah, that's why I find it interesting that the and it's a
bit more of a controversial topic, which is the convergence
or the use of AI techniques for some of these disciplines.
You know, whether just as there's influences of using CFD
or numerical methods for AI, is AI got any use within CFD or or
you know, do you see it in the maths world that the actual use
of AI techniques or is it controversial or not sort of
proven? You know, just as you were
bringing in like techniques from CFD to AI applications in the
finance. So I mean, in terms of AI within
CFD in general, I would say I'm a sceptic, but but to some
extent that may be that I just haven't spent enough time to
actually see what is is happening.
I think potentially the idea of an AI-based turbulence model
might, might make sense. I mean, turbulence modelling is
such a challenging topic and there's so little progress I
think has been made in the last, you know, 25 years that AI may
may be the answer there. I think in I think, you know,
for detailed CFD. I think it it, it's going to
need to continue to be traditional CFD methods.
You know that whether AI can then get trained on the results
of a large number of simulations such that it does a
pretty good proxy for the purposes of of design
optimization. Maybe I guess that doesn't
particularly interest or excite me.
But again, maybe I'm, I'm just showing my, my, my age.
I, I am interested in the impact of AI on research more
generally. So, you know, I'm really very
impressed by the capabilities of the latest ChatGPT, you know,
and you know, I think it really has improved hugely, you know,
over the last two years, say, you know, so the pace of
development is, is fascinating, you know, so, so I'm doing
quite a few experiments in different settings just just
to understand what it can do. So, so you know, I mean, just
yesterday I was getting it to, you know, tell me all about, you
know, Brownian motion, spatial white noise, space-time white
noise for, you know, for stochastic modelling.
And so put together a whole lot of information for me to, to,
to give to an MSc student who's doing a project with me, you
know, all all the way to it, you know, telling me about the
stochastic heat equation, providing me with some sample
code, implementing it in, in a simple, you know, finite
difference approximation. It, it tried to set it up to, to
use multilevel. So understood about multilevel
it it didn't quite get things right, but but it, you know, did
surprisingly well, you know, so I really wonder where we're
going to be in five or ten years time, you know, so.
Yeah. I think it's entirely possible.
I might even go as far as to say likely, that 10 years from now
you will have a proof assistant that is capable of looking at a
theorem and proof given the necessary background
information, and it won't be able to say with certainty that
the proof is correct. But it will on many occasions be
able to highlight bits in the proof that look dodgy or don't
like wrong. So yeah, So I, I, I kind of
think that AI, you know, you should absolutely never trust it
100%. You should always look at what
it produces. But the fact that it gets some
things wrong doesn't matter if it also gets some things right.
And it is, you know, it's very helpful.
I mean, another another thing I did with it a few days ago,
it's always irritated me with programming Intel CPUs that at
times their compiler is very poor at vectorization.
And so if you really want to get performance, you have to work
with the vector intrinsics, you know, the AVX-512 intrinsics.
And it's irritated me the fact that Intel doesn't provide you
with a C++ class that has operator overloading, you know,
to go along with all of those intrinsics.
And so I asked ChatGPT about this and it, it produced me
with, you know, it gave me a C++ class definition with, with all
of the operator overloading, you know, and I'm, I'm not a
good enough C++ programmer that I could have done that
myself. So just saved me, you know,
incredible amount of time. So things that in a sense are
are routine enough for a large group of people now become
automatically available to everyone else who isn't part of
that, you know, So, so as a productivity tool, you know, it
really is capable. So I'm, I'm doing this, I mean,
partly from, for my own benefit, partly also to just spread the
word amongst my colleagues, you know, to give them a range of
examples of, look, this is what AI can do for you.
Yeah. I mean, I agree.
It's, it's amazing what it, it's and and it's, it almost does
need people to evangelise or show it because if you haven't
seen it, you don't know. But once you see that, Oh my
God, this is. Yeah.
So I'm, I'm, I'm now kind of getting into that evangelism
group. Yes, Yeah.
It it it, it is fascinating. So here's maybe a question for
you as, as we get towards the end of this discussion, which
I'm sure we could carry on for hours, because I, I, yeah,
you've got so many interesting things that we didn't talk
about. But I'm I'm conscious of like
your, your time, but maybe more philosophical question, which
is if you were if you were now an undergraduate.
Yeah. In today's world, what would be
your advice on a career trajectory, what to focus on?
Like, I know it's a very difficult question, but seeing
everything you've done, is there anything you would advise now,
somebody who is, yeah, 18, 19 or doing an undergraduate and is
thinking about, you know, academia, industry, Is it good
to work with industry to get that understanding?
Yeah. Yeah, it's tough because I mean,
I think the main advice is to try to do something that you
enjoy doing. You know, you know, you know,
you know, we don't want everyone to be doing the same things.
Everyone should kind of pursue, you know what, what, what they
like. If, if I mean if it was me again
now, you know, thinking of, you know, where I was as a
teenager, you know, I would, I would still probably be, be
heading in into mathematics. You know, I would.
I think by my nature I was always interested in
applications. So I think as a teenager I was
interested in maths because it helped me with applications in
physics that that was kind of the motivation.
I mean maths now one of the areas of growth I think is in
terms of mathematical modelling in medicine.
So, so I do think if, if, if I was maybe an undergraduate now
and, and thinking about where I wanted to head for PhD, you
know, computational methods applied in, in the sort of
medical area, I think may, may be an area right right now.
It's still challenging with the whole funding situation.
So, you know, we have difficulties linking up medical
departments with science departments and, you know, the
whole way the UK funding mechanisms work, but at least we
don't have the problems that, that, that the US has for us.
So yes, I mean that that's maybe what, what I would end up
doing if, you know, I was, I was 18 again.
Now I don't know. But I think the main thing is,
is to try to enjoy what it is you do.
And, and you know, don't don't worry too much about, about the
future, you know, you know, you know, when I started as, as an
undergraduate at Cambridge, I really didn't know what I was
going to be doing at the end. I think, I think now students
start as undergrads with much more of an idea of what they
want to do at the end of it. And maybe there's.
Pressure. Yeah, there is more pressure
now. I mean, back when I was an
undergraduate, so few people went to university that in a
sense you were, you felt, you know, guaranteed that you you
would get a good job at the end of it.
And people do do not feel that guarantee now, you know, so, so,
so I do, you know, I do understand and appreciate that,
you know, I certainly wouldn't discourage people from doing,
you know, you know, pursuing interests in programming.
I know, I know there's talk about AI is going to do away
with all these programmers. No, I don't think so.
I think, you know, you, you, you, you still need people with
with programming skills. I think I would definitely
encourage people to develop their AI using skills, you know,
so and engage with these AI tools, you know, because they
they are hugely useful, you know, and yet at the same time,
you have to have the critical skills to look at what they
produce and ask whether it's right or not, you know, so, so
you absolutely still do need to have very strong understanding
of your technical area, but given that, you know, the AI
tools can, you know, make you more productive.
Would would you agree as well that I mean it's easy for both
of us to say this, but that the if you want to guarantee a job
for the next 60 years, having a maths and/or engineering
background allows you to turn your hand to almost any
problems. And it's, it's quite a general
skill set that is desirable because you could turn to maths,
you could turn to CFD, you could turn to biology, you could
turn, you know, there's always a need for the sort of
mathematical simulation side of things.
I mean, I, I, I think maths is rightly viewed as, as a subject
that has, you know, lots of real world applications.
I think, I think it's viewed more so now than probably 30
years ago, you know, so you know, you know, because of data
science, because of AI. You know, the importance of
maths I think is much better understood.
We still at some time, you know, have have trouble convincing
politicians of, of, of that from the point of view of funding.
You know, we, we, we do feel a bit hard done by, shall we say,
in terms of supporting, you know, you know, the underpinning
maths that's so important for so many applications.
So yes, I think, you know, a maths education gives you a very
firm foundation for, for life. But again, I would, I would
emphasise doing things that you enjoy, you know, you know, I
think, you know, there are people on, on the, you know, the
creative side where I don't see AI sort of taking over.
I mean, whether it's creative in the arts or being a chef or
being a singer, you know, I mean, yes, I mean, we'll see
where, where, where AI gets to in terms of, you know,
generating your pop songs and things.
I don't know, yes. I mean, maybe, maybe it'll turn
out yeah, but people will, will, will still want live
performances and stuff. No.
So, so I think there's a lot of stuff on the creative side which
will will be important for the future.
But, you know, there's only so much that AI can do.
You know, I, I think it, I think it's best to think of it as a
productivity tool that it's important you engage with so
that you you have those skills to be productive.
But don't, don't think that it's going to suddenly eliminate huge
numbers of jobs. Yeah.
Well, and you still you need at least at the moment there's a
huge market for people with HPC, CFD, programming, maths
backgrounds to develop these AI models.
So it's actually a a great time in some ways to have those skill
sets. Yes, I'm not sure that we're,
we're developing enough new people in HPC, you know, and I'm
not sure which degree programs they're coming out of it.
It's definitely not maths. It's definitely not computer
science. Well, not, not Oxford computer
science. You know, places like Warwick
maybe have a bit more or Bristol maybe have a bit more of a focus
on on HPC. Now it's, it's a very actually
good question because I guess by definition, what is HPC is part
of the problem. And I, I would agree with you
that. And again, to link to the AI,
it's even more important because you know, as you know, big AI
training clusters are essentially what people would
have called HPC clusters before. There is essentially no
difference. Networking, you use Slurm or,
now, Kubernetes and things. But but you're right, what
course teaches? Yeah.
Yes, I mean, I know some of the inside story on, on in
Microsoft's development of large GPU clusters and how they in,
you know, involved a consultant who is one of the world's
leading HPC experts, you know, so, so yes, absolutely.
You know, doing, doing these really large systems is is a
massive HPC challenge. It, it, it would be interesting
to know, you know, the people that NVIDIA hires as DevTechs,
what is their background? You know, to what extent have
they been formally trained in HPC or to what extent is, has it
been a passion? And they've learnt on the job in
various application areas and they've, they've proved their
skills and, you know, been hired on that basis.
That's that's a good question. And I think maybe this goes not
full circle, but half to our discussion about some of these
OERC research software engineers.
At least what I see is that people get their HPC skills
often during their PhD when they're using an HPC facility
and they sort of become best buddies with the admin because
they want to get higher up in the queue where they have to
sort of figure out some courses.
And so they haven't done necessary HPC course, but
they've had to do it to get access to the compute.
But somebody was there to manage the system.
So it's probably those, like RSEs, I think they call it.
Is it RSEs now research software?
Engineers. Are sort of like this lifeblood
who support the people and are probably helping them.
That maybe is an underrated skill or need in the community.
But you're right, I'm I'm not sure of any HPC official
training, but maybe I need to do a bit of digging around to we
sort of take for granted stuff that we have.
We never realized that. What if all those people retire?
I suppose is your point, isn't?
It like, yeah, yes, yes, I wonder yes, where, where, where,
where the next generation of academics is, is, is coming
from, but. Yeah, yeah.
But I, yeah, really appreciate you talking.
And I, I will put some links to the the sort of episode notes
because you, you know, I saw you've got a great website as
well where you link some of the, you know, full list of your
papers, some of the courses that you're doing so that people
could maybe read up because we didn't get into all of your
academic papers. But I know you've done a good
job of linking and some of the presentations and things like
that. So I'll put it through.
But yeah, I just want to say thank you also collectively
thank you because all the work that you did in throughout your
career has actually helped people like me and others who
work in industry to be able to do CFD in an easy way that we
take for granted now and don't have to program 2D grids by hand
like like you did. So thank you from all the CFD
people who take it for granted now.
And yeah, thanks for taking the time to to speak to me.
You're very welcome. It's fun.
Fun reminiscing about these these things from days past.
Yeah. Great.
Thanks very much.