You should feel uncomfortable in your job about 30% of the time.
It's a good rule of thumb because if you're comfortable
100% of the time, you're probably sticking to the stuff
you know and you're missing a whole load of other stuff.
Every career move is a trade. You're trading what you have
specialized in before with what you don't know, and you give
them the specialism and then you take the learning.
This thought that architecture follows money.
Where did it originate from? Where's the seed planted?
It's not about individuals doing stupid stuff or being stupid
necessarily. It's the system.
I want people to be motivated to try and change things, but I
want them to understand that it might be that the system is is
rigged against them and not to punish themselves too hard for
that. Today I'm joined by Ian Meal,
CTO over at Container Solutions, and we talk about how you can
level up as Senior Software Engineer.
We cover systems thinking and what that means, and we talk
about how your software architecture actually follows
the money. So enjoy, I do these Q&A's where
people send questions and one of the questions I got recently was
how do I pitch to either senior leadership, technical or
non-technical and how do I get their buy in and the lot.
There's a lot that goes into that, which I think is
interesting because we as technical people, we are in
positions where we can influence certain decisions.
And through doing this podcast, through being involved in
conferences, I feel like there's a lot of skills that you don't
really pick up on, which is quite interesting, like knowing
your audience, for example, if you're pitching to senior
leadership, if you're really technical in your conversation,
you're going to lose people in. In essence, if you're talking to
the CMO versus the CFO, you might hook them with different
ways of communicating, but also in the content that you actually
share, what makes them tick and what could actually have them
have buy in. You can leverage that.
If you read know your audience, that's separate from the skills
to actually communicate that. I think as you're talking, the
thing that I think I'm thinking of most is the because I've done
more presenting the most. I don't know if it's because of
that, but but I've found that I'm much more able to call out
whatever is in the room at the time.
So if I'm presenting to someone senior and it looks like they're
bored, I'll just say I feel like I'm losing you.
Oh, really? Yeah.
Oh, that's nice. Yeah, it it, it, it, it works
because that gives them the space to actually say, yeah,
actually you have lost me. I didn't understand this.
You go, OK, let's go back to that.
And you can might be able to get them back that way.
But you can only do that if you're really confident in your
in your ability. But in in the same way, like I
think a lot of people just want to get through that meeting and
get out the other side. To race through it.
Without being humiliated. And once you're certain
confident, once you're a certain level of confidence, you can,
you can call out those things and, and make them explicit.
I'm, I'm doing that. I'm working with a client, a
customer in a sort of, it's kind of a cultural change program,
but it's focused on specific technical area.
And I'm finding myself a lot of the time saying, I feel like you
don't want to do this. I feel like, you know, and just
saying that you can see the relief when you say that because
then they can say, yeah, actually I do feel that because
of this. And then you can go, OK, let's
talk about that. That's kind of fun.
That's really strong. Yeah.
It's one of those skills that I
feel like if you're talking about seniority, at some point
you're going to be comfortable within your technical domain to
a point where it's so comfortable that you might not
want to try anything else. But that's besides the thing.
I do feel like if you go from a senior position either to a
leadership track or definitely to AC level, even if we're
talking about ACTO as a label, you will need to be well
equipped to have conversations like that in front of an
audience, but also one-on-one to make sure you actually find what
is what makes people tick and what makes people either object
or how you get the buy and you need.
Yeah. And I think that that point
about getting outside your comfort zone, outside the domain
that has made you successful in your career is an important one.
I I like to say to people, you should feel uncomfortable in
your job about 30% of the time. It's a good rule of thumb
because if you're comfortable 100% of the time, you're
probably sticking to the stuff you know and you're missing a
whole load of other stuff. Yeah, you're coasting.
You're coasting. Yeah.
And you should, you should be exploring those areas that that
make you feel a little bit unsure of yourself.
One of the best things I ever did was I worked for as a
company for 14 years and I knew that domain like, really well,
as well as almost anyone. And then I went into a
completely different domain. I mean, not completely
different. It wasn't like going to, you
know, doing sociology somewhere on the other side of the world,
but it was, it was a completely different kind of company,
different scale, everything. And, you know, I was completely
lost for a while. I just didn't, I remember asking
my, the guy who hired me, asking him questions and could see the
expression on his face was like, oh, he really doesn't get what
he's into yet. He's just thinking about this
the wrong way. And he didn't, he didn't, he was
very kind. He didn't slam me.
He just kind of showed me where I what I was missing and, and
you know, once I got through that stage, I realised what what
I was in and, and learned a lot from that process.
So yeah, I think being uncomfortable is something you
should check on regularly. I think it's really cool that we
can, we can leverage a very technical or very hard skill and
we can still be effective in a lot of domains.
So then the fresh part could be I'm going from something that's
very retail, very consumer facing to something that's
completely financial and operating towards businesses,
which domain wise there might be overlap, but in the most cases,
there's not much overlap. And then what you bring is like
your hard skills to the table and you can familiarise yourself
if you like that curiosity wise as well.
I think a lot of domains are interesting.
You can do that. Yeah.
And it gives you more options in your career.
So the more, I mean, every, every career move is a trade.
You're trading what you have specialised in before with what
you don't know. And you give them the specialism
and then you take the learning. And if you keep doing that,
you'll end up with a whole load of arrows in your quiver that
you can then you can then use later in your career.
So yeah, really, unless you're like a hyper specialist and
those people are very rare. They exist, but they're very
rare. I recommend for the vast
majority of people just go get go get new experiences.
Yeah. I could see how how you can
leverage those experiences and it makes you a more well-rounded
individual. I'm very curious from your
perspective with either seniors you've worked with or from your
own experience, when people reach a certain level of
seniority, some people kind of get stuck with regards to growth
and they experiment a lot, but they might fall back on that
comfort zone that is more the hard skill side of things.
What is your advice for people to actually breakthrough and go
beyond a certain level of seniority?
From my experience and I, I've mentioned it already, just
change your environment completely because another
aspect is not just that you are more comfortable, but everyone
else is more comfortable with you being in that role.
I noticed that when I moved from that, I was in that company for
14 years and that company went from 30.
I was 39th to join. We went to 770 people.
So it felt like many, I did many different jobs in that company.
It felt like many different things, but I was, I was Ian and
you know, people knew people had me in a box.
And when I moved to another company, I noticed people were
listening to what I was saying for a, for a time, you know, for
six months, people would actually listen to what I was
saying in a way that they didn't before.
And then had a bunch of other challenges because of that.
But it allowed me to break free of that, that comfort zone, that
domain. And then once I'd done that
once, it made it a lot easier to do it subsequently.
But, but it is partly definitely that that people will see you as
that person. I know many people who are in
certain jobs and they want to, they want to progress, but
they're just seen as a techie and, and they just, yeah, they
just get put in that box and they're useful and people just
want to use them for that. Yeah, 1.
Of my previous guests, Pasha, he used to work at Uber and he said
that's one of the reasons I left.
He called the the burden of context just due to being there.
Historically he was being dragged in sessions to explain
the context and people would also come to him for advice, but
he wanted to focus on very different things than what he
was actually focusing on. So then by default, through you
being there, also probably being good at what you do, you get
this burden of context and people have expectations of you.
You're known within the org, but it might not be what makes you
happy. Yeah, and it, but it, it comes
with great benefits, right. You're just, you're very secure.
Yeah. Like, they won't fire you if you
know a lot. That's true.
And you're still, you know, still motivated to work and not,
not doing anything stupid. You will, you will keep your
job. And that's, you know, for me, I
had kids in the middle of that time, and my wife chose to stay
at home. We chose for my wife to stay at
home with the kids. And that meant that there was
more pressure on me. You know, I felt more pressure.
And so I stayed a lot longer than maybe I would have done
otherwise. And one of the reasons was that
feeling of security. Exactly.
Switching concepts is is very powerful.
I feel like I I do it a lot in consultancies.
For me the downside is I have to build up the relationships and
the people like certain political credits and
relationships all from scratch again now to the point where I'm
actually like, I don't know how long I would want to do that
because I feel like being in an org on the longer term also has
its like. Its benefits, I love that.
I mean, I wrote a piece once on why I love consulting and one of
the reasons was you get to meet new people all the time.
And I, I really enjoy that. And you start to see patterns,
but nowhere is the same. It's like being a doctor, I
guess, you know, no, no two patients are the same.
But, but you do start to see patterns of, of behaviour and
patterns of, you know, thoughts and and and so on.
And that's that's where the book came from, I guess.
Yeah, Yeah. We talked about seniority and
moving to beyond seniority, breaking through a seating in
essence. Can you tell me more about that?
I think there's, I mean it goes back to what I was saying
earlier, I guess about the, the, the adding context.
I think it becomes easier as you get older because you've had a
more of a varied experience. And I'm not just talking about
career, but also, I mean, when I, so I'm CTO now of container
solutions, which is a cloud native consultancy and we help
businesses with really challenging organizational
change, organizational technical change problems all together,
all in one. So we don't, we talk about
strategy, but we also do do the do the tech And I joined as an
as an engineer. I wanted to get back to
engineering. You know, I had that sort of
kind of midlife crisis of like, what am I now as in my career?
And I was like, I want to get back to engineering.
That's when I was really happy. I guess I felt that urge to I'd
had AI had a kind of, I suppose it was a bad experience, bad
career experience and I kind of want to get back to my comfort
zone. But I found that as soon as I
joined that company, I was asking about the balance sheet
and you know, what's the, what's the cash flow looking like and
where's the graph of, of, of revenue versus profit and so on.
And everyone's like, no one else is asking these questions.
You know, you should think about going more commercial, being in
the more commercial side. So I then moved into into sales.
So I was still with customers, but I was also heavily involved
in, in the sales process. And I really enjoyed that
because I got to meet, you know, all the, all these clients were
coming to us talking about their, their challenges.
And I enjoyed meeting them and I enjoyed talking to them and, and
what I yeah. And, and so I love that work.
I still do, I still do that. But it wasn't, it wasn't only
what I did, but through doing that, I then I saw the
commercial side more more clearly how, why people buy
services like that, like the ones we had and how those things
get sold and then how they, how they get handed over to
engineering. And I guess through that I got
to know the, the board. And then the board said, you
know, you've got to be AC if you want to sit on the board.
So you're now CTO. And so I didn't set out to rise
up the, the hierarchy, but I just found myself because I was
in this new situation and I'd seen other situations before.
I wasn't just interested in, in the engineering.
And I still miss it. You know, I still don't do
enough of it as I'd like to, but I find that as I get older and
this, I guess this is another aspect of it.
And, and I've had more experience, you know, I've got
30 years in in the field now, as you know, 25 years in the field
now. I find that my value is far more
in recognising what works and, and having the confidence to
say, no, this is, this is the right person to hire.
This is not the right person to hire.
This, this engineer is going to work really well with that
client and not with this client. Those kind of judgement calls
are seen as more valuable than than before.
I mean, I was, I was reflecting to my, my CEO the other day that
a lot of my job is now simply coaching.
It's calling out those meetings where, you know, no one seems to
be talking. Why is no one talking?
That kind of conversation is far more I do now than technical.
Now I have people we, I've hired people I trust to make technical
calls and I, I don't, I might disagree with them, but I don't
really ever overrule them because they have to live with
the consequences, their decisions.
If it's a, if I think it's a really bad decision, I might, I
might do that. But generally they're not,
they're not obviously bad decisions.
They're more like, well, I prefer this, but sometimes I'm
just too old. Like I'm not, I haven't been in
the field in the last 10 years, so maybe I'm slightly out date.
Had an argument with one engineer recently about Orms,
and he was in favour of using them in this particular context.
And I'm like, I don't like them. But, you know, maybe that's
because I'm old enough to remember writing all my sequel
by hand and. And, you know, not just not
wanting to let go of that. So.
Yeah, but. But I think also because if you
have grey hair and you're a little bit older people in
positions of seniority and authority, just trust your
judgement more. Yeah, you, you tend to make, you
tend to have, I guess with age you tend to have more, you tend
to make less impetuous decisions, you tend to reflect
more on your decision making. So yeah, I think partly just
getting older helps, but but having that that wider set of
experiences can help too. Yeah.
I, I learned a lot I think when I moved away from software
engineering to actually being hands on responsible for
product. Because then I also got to touch
a lot of other aspects with regards to, OK, what, how do we
do marketing or even how do we do more sales from that aspect
specifically or what certain decisions do we have to actually
make? And how does that impact tech?
And how do we balance then delivery versus quality?
And do we need to cut some things when going to production?
And can I actually communicate those trade-offs well to get buy
in? You're a product manager, Yeah,
right. So I think we used to call those
project managers back in the day, but now we call them
product managers. I mean, it fills the same
position in in the in the system.
And yeah, you, you can't just think about the technical,
you've got to think about your stakeholders and the road map.
And did, did your key customers want this thing?
And do you have to sacrifice that other thing for it?
Those, those kind of decisions are really, really tough.
I mean, my, my first job was not dissimilar.
So my first proper job in IT was working for an online gambling
company. So it was sports betting back
end systems and the structure of the business was that you had a
tech lead on a particular customer.
So it was a product, but, but we had about 6 big customers and I
had, I was one of the tech leads for the big customer.
And you were basically the, the project manager, I guess, or
project leader or, or however it was.
And you had a project, you had a project manager with you who
was, I hope I'm not upsetting people, but I think they were
slightly subordinate. So the techie was in charge, but
the project manager was kind of probably the wiser old head who
would help you probably navigate some of the things that as a
young techie you were you were not so greater.
I mean, that was kind of what happened.
And also they were more organised and the customer would
look to them for the more organizational side.
They would look to you for the kind of, you know, are you going
to break production this week with this release or not kind of
questions. So so yeah, I learned a lot, I
guess, about balancing those things because I'd have
engineers saying, ah, but this is the right thing to do.
And you'd say, no, we this, this, this release can't fail.
Like, we're in trouble already for the last release.
So I want a safe release this time.
And they'd be like, yeah, but it's the wrong thing to do.
And, you know, you know, I was not prepared to have.
I was not well, well prepared or mentored to be that role.
But it did teach me a lot about being in that role.
Yeah. I think I learned a lot about, I
think 2 aspects of things. One, what people are responsible
for. That's exactly what they're
going to look look at. So if I'm having conversations
with legal and compliance, they are there to mitigate risk or to
kill it. The best is if there's not no
risk, because if we're in a bigger organization for some
aspects, especially tech, we don't need to afford risk.
So we need to squash it or accommodate it in any, in any
shape or way. So if I did my homework, I can
say, well, these are the risks that we already identified and
these are the mitigations that we have in place.
This is already in place and this is what we need to build if
we want to go live with XY and Z, which is that is very
interesting. But then secondly, it's also how
people are incentivized. So either people have certain
KPIs, and if we set KPIs, we can kind of distill and influence
behaviors. Because all of a sudden if you
get a bonus or you don't get a bonus, yeah, you're going to go
for that bonus. So you're going to actually do
what the KPI is there for? In my experience, techies really
clash with people whose whose job it is not to take risks.
And I come across this all the time because because as a
consultant working with companies that are trying to
change, there's an element of risk and innovation there that
most companies are really averse to.
But the engineers I've worked with in the part, well, pretty
much all engineers I've worked with get really frustrated when
they deal with someone who works in security or someone who works
in legal who are only focused on not making a mistake.
And as an engineer, you're always risk.
It's quite an entrepreneurial job, like you're always taking a
risk. You're always, you know, even
when you trying to code a fix, you could spend a lot more time
checking that the fix is correct, or you could just run
it and see if it works right. And that's a trade off that you
make all the time. You're always, even in that
situation, you're taking a risk and you're always making these
judgements. And if your job is never to have
a security breach, then you're going to say no to everything.
Yeah. That's the easiest way.
It's the easiest way and so like you say, once you understand
their motivation, where they're coming from, what them
incentives are. And then you can kind of deal
with them in a way that they understand, which immediately
puts them at their ease. If you're saying I've identified
all the risks for you, they go, great, you've saved me.
You've saved me the time here. Have you thought about those
risks? Yeah, We've thought about this.
And here's how we've mitigated and here's what we've done.
And suddenly the past becomes a lot clearer.
We actually, Container Solutions is working on a project, an open
source project called Continuous Compliance Framework.
Yeah. And because we worked on
multiple projects which in regulated industries, which
involve spreadsheets and PDFs, and you know, you write down how
you're mitigating a risk and how you're the evidence you've got
to conform with this thing. And then every six months
someone comes along and does an audit on you.
So we had this experience over and over again.
We said a lot of this can be automated, but anyway, so we
produced this, this, this product that that allows people
to manage that and we open sourced it and we spent a lot of
time working on it. And a lot of the value of that
is simply that if an auditor comes along, they see that
you've got your house in order. You know, it's not that your
software is necessarily more compliant than other software,
it's just that you're you're able to expose it more honestly.
And I suppose that one thing I see a lot in engineers is this
inability to see that because something is objectively
correct, that's not enough. You have to communicate that in
a way that that people understand and that that is a
whole job in itself. And many of the techies I've
worked with have got frustrated with that because they don't,
they don't see why should I have to do that if I've done, done it
correctly. It's a very, it's a very common
pattern that I've seen. And it's, it's, it's something
that some people never want to do, but some people can actually
get a lot better at once. They realise that part of the
job is that is that is doing that.
Yeah, I mean, there's there's organisations where you don't
have to do that as much, where more risk is just part of the
game, like a start up, for example.
And I especially early on in my career, I did not have any hate
space to think about any of this.
But as, as you get more comfortable, I feel like there
is room for more learnings and more opportunities and you get
to experience and explore. I'm I'm definitely a person that
doesn't like to have this thought of like, oh, woe is me
if there's a legal department and they always say no.
I would do my best and try kind of everything in my ability from
my end to navigate that structure.
Because if in the end, then my conclusion is, well, in this
organization, it just really doesn't work.
I at least have the feeling I tried everything.
But if I haven't tried everything, I can't just make
the conclusion that this is a department where no is just what
is happening. And I feel like with engineers,
sometimes even with tooling, it's like that is shit that's
always there. It's very black and white.
It's very dogmatic. Yeah, I think what you said
about the start up is it's a really important point because
if the start up is completely different, the the, the pressure
to deliver is completely different.
I mean, in the book, in the following the money book, I talk
about patrons and patrons. It's similar to customers.
It's not the same concepts, but if the start up, the patronage
system is much simpler, who, who is funding the business, right?
That's one thing. And then who are your early
customers? They're your patrons, right?
And that's a relatively simple word to navigate.
And what they all want is fast delivery.
So that's what you give them when you get to a large company
like so I went from 30 nights to join in a relatively start up
mode to 120,000 people at at a bank.
Oh. Wow.
Yeah, and, and suddenly everything's on a different
scale and I'm dealing with a legal department who always just
says no. And one of the things was one of
the things that happened was I ended up realising I'm in the
wrong company. Like it's not, it's kind of,
it's not me, it's them. Or rather it is me because I, I
like to be entrepreneurial and I like to take risks and I like to
do things. I don't belong in a big, big
organization. And one of the themes of the
book is, you know, if you're, if you're a techie and you're
seeing that the technology isn't what you'd like it to be like
the architecture is wrong, whatever, you might end up
realising that the structure of the business is not going to
facilitate you doing the right thing.
In fact, it isn't the right thing necessarily for that
business. So, you know, realise that,
understand that and decide what you want to do about it.
You can seek to change it from within, but you're probably
going to have to to a large extent play within the rules of
that game or go go somewhere that is more suited to your to
your instincts. Yeah, has has different rules
for the game completely. Yeah, I think it's very hard to
change an organization of like 120,000.
I do think that especially nowadays with AI system
development tools, the build is really accelerating the build
phase of something. So we can create tools and we
can create software at a speed that is unprecedented and people
have opinions on the quality, but let's just put that aside.
If we build more and more and more and we can create faster,
for me, two things are going to be very interesting, figuring
out what problem we're solving. So the right problems.
But then also, once you have something that you believe
solves the right problem, there is still the huge challenge of
distribution and adoption. Because the fact that you have
one tool probably means that you've solved the problem.
And if it's a really good problem, then you will have
other tools that are very similar that are trying to solve
the same problem. It might be a red flag if you
have the only tool that solves this problem, because then it
might not actually be a problem. But let's say you found a
problem you want to solve, then how do you actually get into the
hands of the users and accelerate from that process?
Which I think is interesting. Yeah, the cost of the cost of
engineering a solution to a problem has gone down.
Yeah, I often think of is is it Jevons paradox?
I wouldn't know. I think it's, I I'll check
later, but I think it's, you know, this story where like, OK,
in the UK, in London, they built the, they built a ring Rd.
around London that was designed to alleviate traffic and they
built it. And then traffic was worse than
before because you build more infrastructure and more people
use it, more cars on the road. It just got worse.
You just had another place. And I think the same is true in
San Francisco. Is it San Francisco, CA where
they've got like 8 lane motorways and they just add
another lane and then that lane just gets yeah.
So the paradoxes you think it's going to alleviate the demand
actually goes up when you create more, create more capacity.
We now have a greater capacity to write software.
So more software is going to be written.
So there's going to be 10 solutions to every problem.
So, yeah, like you say, the the challenge is going to be how do
you differentiate yourself from the competition?
You know, and it may be that people who don't use AI to
produce a better architecture solution, which scales better,
or it may be that, you know, it's, it's all about the
marketing and and so on. But yeah, it's, it's I think, I
think the economics of producing software.
I mean, I remember when, when I started in the industry, there
was there was no, there was not really even search engines
weren't that helpful. You had to get a book.
OK, if you only learn Python, you pretty much had to get a
book. That's how, that's how you
learned. And the other year when I after
3.5 of, yeah, I think it was 3.5 came out, I used it to build a
React app myself. And I'd, I'd kind of tried to
read books on React and kind of couldn't, it didn't stick or,
you know, I just didn't, didn't find it worked.
But this project helped me, helped me learn how React
worked, you know, from day one. And it was perfect because you
know, I would ask questions like, oh, is this, is this a
single threaded app? Is this, you know, how does this
work? Is this asynchronous?
And because I had that experience before, it made it
very easy for me to learn. But you know, you see other
people use AI in a different way.
And, and I just think about the, the maintenance challenges of
all this stuff. There are going to be businesses
which have been built on, on these technologies which no one
knows how they work. And then they find that they've
they've hit a limit of what can be done.
It's reached a size and so someone has to go in and unpick
it. I think that that's going to be
an interesting, another interesting challenge.
Yeah, there was this, I think it was a few months ago, this
person, that vibe coded an app. It was kind of a social media
thing where people uploaded pictures.
And I think it was either for forming friendships or
relationships in some form of way, specifically within a
community. And then there was a data breach
and people were like, well, surprised because this person
was preaching to the choir about how it was vibe coded and how
fast it went to market and how quickly it got users in usage.
And now it's I don't think it exists anymore due to that
breach specifically. Yeah, I'm, I'm thinking, you
know, we're talking about, you're talking about a harness
there. Like there's a there's a
compliance and security harness that must be around all these
projects, right? And it sort of makes me think of
that. There was that C plus was it the
C compiler story? And people were like, wow, it's
written, it's written AC compiler in Rust using just.
Is it 200 hours? Or two from Anthropic.
Yeah, I can't, I don't know the details, but I, I, I read a very
good piece on it which said that the actually interesting part of
that was the test harness. And it was a test harness that
that was really beautifully done that allowed the AI to, to, to
autonomously build, build the solution.
And building a test harness is really hard.
Yeah, absolutely. And maybe you can get an IAI to
help you with that. But, but, but that that
building, doing that right is going to be a real skill because
you'll also have to think about security compliance, you know,
maintainability. These are all the things that
that affect the cost of, of running that software.
So. Yeah, I, I had a certain
perspective, I think on things like security and compliance,
especially within software engineering.
I thought maybe that's kind of boring.
And then I talked to someone in big tech that is responsible for
that. He's, I think one of the staff
engineers. So he influences a lot of
decision making on that. And he gave me a completely new
perspective. He said that in essence with
software is very great, right? You create something new
functionality, your user can have an opinion on it.
It's more subjective, can be hit or miss, but he says when it
comes to security and compliance, it's a lot more
black and white. So either we do it and we do it
the right way and we are not at risk.
So the risk is mitigated or you're at risk, which means that
probably you, if you're responsible, you missed
something, right? So there's no or there's way
less margin for error. And then he said, for me, that's
software engineering at a way higher level than when things
are more grey, which I thought was interesting.
I. I have a different perspective
actually, which is that. So when I, I'll give a bit of
background. So when I worked at this this
bank, we were looking at various tools around security.
So I was, I was in, I was the owner of the Docker product
within this bank. Cool.
Yeah, it wasn't really cool. It was, it was kind of a it was
a kind of a, it was an interesting experience, right?
Because what that meant was that it was actually quite well
organised. Every technology had an owner
and that person was the point person for that technology.
And I was the point person for Docker because at that time I'd
written a book on Docker. And so they said you.
Have yeah, they go to and I was like.
Great. Like and they were like, ha, ha,
ha, he's got that job now, ha ha, you can deal with it.
And what that meant was then if anyone wanted to use Docker,
they would phone you up. Really.
Yeah. And say as a single person.
Yeah, it would just say, I I want to use Docker, you're
listed as the owner. Get me docker.
And I had to explain over and over and over again that, well,
Docker at the time gives you root, so sorry, can't happen.
Yeah, you're the department of no.
I was the department of no, yeah.
So I was put in that position, right?
And, and I understood the context.
So to me, it was like I'm just being perfectly reasonable and
people were not happy with my response.
So it was interesting. I was suddenly thrust in this
from this position of like, hey, let's use Docker.
Docker's great and Docker can reduce the cost of development
and do all these great things. I'd done that in my previous
company and now I was in the position of saying no.
Maybe that actually contributed to the to the mindset that I
have now, which is kind of more tolerant because suddenly, yeah,
I was, I was having to say no because I was like, I cannot do
that. Like even I do want to do that.
I'd love to give you give you that.
Anyway, the point of this story is that as part of that, I was
heavily involved in the process of deciding which vulnerability
tool to use, vulnerability scanner to use.
And I remember having this kind of moment where I, we were
looking at 3 or 4 different options and we had all the
outputs of them. And I was like, they all
produced similar outputs, right? And I was like, the problem here
is not that any of these tools is bad or good.
The problem is that I, there's like volumes and volumes about
how are we going to manage this? And this is where it didn't, it
became not a black and white problem, right?
Because for each of these things, you had a number, it's a
security risk number, right? And then you had to look at how
likely is it that applications are, are going to expose those
vulnerabilities. So you know, some, some
vulnerabilities like 9, but if they're never exposed to the
Internet, it's no problem. Or if you're not using a
particular operating system, it's no problem.
So it wasn't a black and white thing like you had to actually
think like, what is the process for deciding which of these
things we should focus on? What is the risk benefit of, of
tackling them? So I that from my perspective,
that was, that's the difference to what what you describe.
I mean, if you have a number against a thing and you can say
you've got that component in your, in your, in your Docker
image, therefore, therefore you're exposed, go fix it.
I mean, that's, that's true. But the reality is that not
everyone can go around. I mean, if everyone went around
fixing all their Docker containers at the time, no one
would have got anything done, no?
That's why I think the best people in in the positions are
responsible for that also needs to have or need to have a
mindset that aligns with what the company is trying to
achieve. So if you're building software
as a service, one of your core components is still very much
feature delivery and tailoring towards an end user.
And if you're then responsible for data privacy and security,
your margin of error is is very low to a point that at some
point you cannot afford to have errors, but you still want to
have the mindset of, OK, these are the things that we have to
accommodate for, but we still want to deliver as fast as
possible. Then it becomes interesting.
You mentioned customers and I would again, I would move that
out slightly to patrons, right, Because if a regulator, for
example, is a, is a big patron, maybe the biggest in, in a bank,
it's the biggest patron there is, they can take away your
banking licence and then you don't have a business at all.
So understanding your customers is very important, but also in,
in and again, in, in a, in a bank's case like the customer
is, I don't want to say less important than the regulator,
but but you're thinking about the regulator first, because if
a customer leaves, it's not, not that big a deal.
And if you get to retail banking, it's the regulator and
individual customers don't don't matter.
If you get to Wells banking, it's a bit different each and
that's often actually why you see in banks, wealth banking
applications are more bespoke and less kind of off the shelf
generic tooling. They tend to have been written
because someone, some group of someone or group of customers
who are very important to them wanted something a certain way.
So you want to be able to change that.
And it's a very good example of how the money, the architecture
follows the money in a retail bank.
You just want off the shelf tools that are like volume
centric in a bank. You don't.
Technology isn't often a huge differentiator.
People would argue with me about that, but but basically you want
something that's safe and reliable and just runs.
And so if you have millions of customers, you're not going to
listen to your customers in the same way as you are a wealth, A
wealth management part of a of a bank.
This thought that architecture follows money.
Where did it originate from? Where?
Where's the seed plant? So the seed was planted when I
did my first degree I guess. So I studied history at
university and in the UK the study of history is essentially
a degree where you have to argue cases.
In fact, law and and history were the War One thing.
And the thing about the study of history is that if you're asked,
you know, why did the why did the Second World War happen?
And if you just wrote an essay saying it happened because of
Adolf Hitler, you'd get very, very low marks because you
haven't considered the systemic causes.
Not that individuals can't, can't, can't affect history,
but, you know, they don't operate in a vacuum.
So historians are systems thinkers from the ground up.
They're thinking about, you know, economic production.
They're thinking about commodities, you know, mining,
how money, well, how money moves around the world and so on.
And so there's this joke that all historians are Marxists now,
which doesn't mean they're left wing.
It means they think of things in terms of economic forces.
So economic forces shape things. And that's that's that's why
things happen. And, and so this sort of
systemic thinking, I was already interested in Florence
University, but it really cemented that, that way of
analysing a problem. And so by the time I got to
working at, at a bank, I was ready to understand this.
So I worked on the Openshift, the Kubernetes platform at, at
this bank. And this is 2016.
So it's the very early days. I think it was version, I think
we went to production with version .23 point 2 of
Openshift, which is like 1.2 of Kubernetes.
One of the big challenges we had, this was kind of the first
pattern that I really kind of developed was that usage of this
platform was exponentially growing, right?
Every three months it would double.
People would just people really wanted to use it.
And you'd think that if your users user base double s every
every three months, you'd get more funding to but no.
And you know, this wasn't used to people around me, but I was
thinking, OK, so so why are we not not getting more funding?
And the reason is that that we were in a business unit that was
a cost centre and that business unit got its funding every year,
argued for it every January. And there were fights at board
level and eventually they'd get their money.
And then the money was allocated to the different teams within
them. We were just one team within
that. And if we did super well and got
doubled our user base, there was no metric to to allocate funds
to us. But then what's the incentive?
Why do people do their best? Because we're morally superior.
OK, that's fine. That's good.
But like to double? Yeah.
Yeah, why? Why?
Why did we make the effort to do that?
Well, from my perspective, it was the guy who hired me.
He was really motivated to solve this problem.
He loved building teams to solve problems like this and and.
He's he, he worked at this business for 30 years.
He's actually a colleague of mine now and brought him with.
Me. That's nice.
Yeah, Yeah, really cool and. He's a, he's a wonderful guy.
And, and, and he, he, he did that, I think, I think why did
he do that? I don't know.
It could be intrinsic motivation.
I think it was just intrinsic motivation somewhere.
Yeah, I think it that's what got him out of bed in the morning.
It's the thought that, you know, he could see conceptually why it
worked out. Interestingly, he wasn't a sort
of coder by background. He was, he'd worked in in IT for
a long time, but he loved working with techies and he
loved enabling them. And he was a great, great sort
of leader of that group because he navigated a lot of the, and
he still does this now. He navigated the governance and
the regulatory side for us and he actually got it done.
It wasn't the techies that really got it done.
It was, it was kind of him who who really spearheaded it.
But yeah, I mean, now I would say his, his interests, his
motivation was misaligned with the organization.
And one of the things we tried to do was to see if we could get
a way of moving the funds from the other business units who
were consuming this central platform to fund us so that
every, every team who uses it pays for the centre.
We didn't get anywhere and I wasn't in the the thick of that,
but I understood that it's very hard to get in an organization
like that gets the people who make those decisions and and get
them to change the way the finance work.
I mean, it's just, it's very hard.
It's very hard. It's very hard.
It's like this is the way we work.
Like we're not going to change it.
But that was my first kind of like, oh, right, we're doing the
right thing technically and we're doing the right thing.
Like I think we're doing the right things and we we're
obviously successful, but and, and the pressure from the other
business units was enormous. And we used to say to them, OK,
do you want to give us two of your staff?
To to help us, yeah. And they were like, no, that's
why should. We do that.
Yeah. But yeah, so, so that was that
was the first time I really sort of saw it.
And then I looked back at my previous job.
So I went for that, that online, online gambling company, we had
what I like to call the exact wrong number of customers to
build the right software. So we had like 30 customers of
which six were really big. So 6 was like 80% of our income.
Oh wow. Yeah.
And they were really, they were obviously in competition with
each other. We were the only back end game
in town and they were really in competition with each other.
So they're the leadership of this six companies said we want,
we want a, they said to our management, we want, we want a
product. We want basically like we don't
want to spend all this money doing upgrades.
Do we just want a product, we pay for it every year and it's
predictable and so on. And our management said, OK,
great, we'll go build products. We went to build those products.
And then the businesses we worked with on the ground, they
didn't want that because they were in competition with each
other and they wanted their thing in production.
So they'd get their bonus or whatever it is that they, they
got and they, they were just weren't interested in, in, in
the, the challenges of, of, of platform, sorry of product to
delivery. They just wanted their thing.
So you ended up with six different code bases that were
broad. Well, I mean, they weren't,
there were six different implementations.
So there was shared code, there was code that was specific to
the client. And then the the database
schemes were roughly similar, but not exactly the same.
And of course this meant that if you wanted what the other
customer had, there was an integration cost.
Oh no. Yeah.
And why is that?
Because the patrons of the business, the big payers, they
they wanted things quickly. They didn't care about.
I mean, online gambling was a very unregulated industry at the
time and it was a very risk taking business.
Because if the World Cup is coming up and you have a, a
promotion in your retail stores for the World Cup and it's tied
with what's going online and you're a week away from the
World Cup and the software isn't ready, just get it out there.
And, and I literally was sat in meetings where people would say,
oh, it's fine. We'll we'll put it in production
and then we'll let support deal with it.
Of course. Yeah.
And I, I was in support, so I moved to support.
It was, it was really well paid. I had kids and it was regular
hours and there was overtime. So I was like, I'll go to
support. No, on call.
It's on call. Yeah, yeah, yeah.
And I, I really enjoyed it, actually.
For a pure techie, it's great because you just see the
problems. Someone brings you a problem and
you're like, how are we going to solve this?
I loved it. I absolutely loved that work,
but it was quite stressful. It was very stressful.
Oh yeah, I can relate. I started in operations my
career completely. Right.
I mean, it's a great, it's a great apprenticeship because you
you're at the sharp end, you're seeing how things actually get
done and the compromises that get made probably helped you in
the product it. Definitely did.
Yeah, yeah. Not just product, but also just
the engineering part of what we need to accommodate for
operationally. Yeah, I mean, yeah, so I mean, I
love that job for lots of reasons, but one of them was the
way it changed. Like I, you asked me about
contrarian viewpoints, right. One of them was we, we used to,
people used to say, oh, the engineers should write the
documentation for support. And I was like, no.
That makes no sense. No.
Support should write the documentation.
And so I'd sit with these really smart people and they would say,
Oh yeah, the really interesting thing about this project is this
engine I wrote that, I was like, I'm pretty sure that that works.
And even if it doesn't work, I'm not going to be able to fix it.
No, but I know that split brain problem is a thing.
And they were like, oh, that shouldn't happen.
Of course. And what happens?
Like 2 months later, we have a split brain problem.
And so we were like, no, we, we will, we will write the
software, you will explain to us what the software does and we
will write our own documentation.
You will check that documentation and then we will
use that documentation for, for, for support.
We call IT support. Then they call it SRE now.
I mean, you know. It's evolved.
It's evolved, yeah, But we were, we were former tech leads and
very senior engineers who moved to support because it was a fun
place to work, believe it or not, We were empowered so I
could fix stuff in production without telling anyone, and then
I would go check it into the source later.
So it was, it was a very responsible job with a lot of
freedom. But you had the challenge of of
these, you know, the very demanding customers because the
site was down. They were losing money per
second. So there was a lot of pressure
to get it back up. In the teams that were
developing the software for the patrons that you mentioned, you
mentioned 30 customers, six of which brought in 80% of the
revenue. How many people actually were
aware of these revenue streams and the impact that it had on
them, the decision making that was being done or how many
people were just there and they got requirements and they
executed? I wouldn't want to answer this
for them, but my impression was that people weren't thinking in
that systemic way and people generally don't.
They think it's because of the people around them being stupid
basically. Which is a common pattern
actually, yeah. Yeah, and, and one of the nice
things about this sort of these ideas I've written about and
developed is that it helps you understand it's not about
individuals doing stupid stuff or being stupid necessarily.
It's the system, like hate the hate the was it hate the don't
hate the player, hate the game kind of thing.
So I don't know how aware they were of it, but I remember I
started to say like, we have the wrong number of customers and
people would kind of laugh and go, yeah, I guess so, yeah,
yeah. I guess the problem with that
thinking though is that it can make you quite defeatist.
I agree and I really don't like that.
Yeah, yeah, yeah. There was this quote that Gregor
Hope shared. He shares it in his trainings
and he says if decisions from the top don't make sense, they
know something you don't. So rather than being they're
idiots, they're highly competent people.
They got into a position for a reason.
It is very unlikely that they just throw a dart on a dog
burden, whatever it lands, that's the impact on you.
So try and make it explicit, right?
They know something you don't, so figure out what that is.
There's a story I like to tell, which is kind of an amalgam of a
lot of these stories. But there was a, a database
table we once worked on that had a column.
And this column was used in a really crazy way.
Like it was in this context, the, the column name was
completely unrelated to how it was used.
It was overloaded. So depending on the context, it
would have a different meaning. It was a single, a single
character column, but it had a different meaning depending on
the context. And we had all sorts of support
calls about this because, because engineers missed this
nuance and it just looked like a really dumb decision.
And it turns out that the origin story of this horrible mistake,
which looks like someone was being a complete idiot.
It was a week before a key key accounting date and something
had to go to production. And if in this context, putting
something in production was much easier if you change the
application code than if you change the database.
Database schema changes required and, and an outage.
And so for very good business reasons, they made that decision
at the time. Now you can argue about whether
that was the right decision in the long run, whether it costs
money, blah, blah, blah. But it was rational actors
making a judgement call at the time.
And they probably figured, oh, someone will fix it later,
which, you know, how many times have we heard that?
I think you should have an opinion on the other will fix it
later. But yeah, the fact that they
made that decision with that context, that's fair game.
Yeah, I mean, and, and this this pattern happens all the time.
And, and it's so easy to say, oh, I wouldn't start from here.
Or, you know, this is this is, this is, this architecture is
wrong. And it's, we should, we should
correct it. It's like, well, OK, how are we
going to do that? Who's going to pay for all that?
So, so, yeah, I, I, but, but you're right.
Like being understanding why the system is working against your
instincts can make you defeatist, I think.
I think I want people to be motivated to try and change
things, but I want them to understand that it might be that
that it's, it might be that the system is is rigged against them
and not to punish themselves too hard for that.
I had one job once where the, the, the, the, I worked there
for a year and the project was a complete disaster.
And I remember really, really fighting it.
And eventually I woke up one day and thought, actually, why am I
trying so hard? Why am I killing myself doing
this? I'll just stop.
And I did. And like 3 months later, the
project was canned. And, but for those three months,
I kind of had fun because I worked, I attached my someone
myself to someone who still believed in it and said, how can
I help you? And he was a, he was a great
techie and he, he had me working on all these things.
Like that's where I learned flux.
And I just got back to the desk and just studied coding and, and
I was like, this is going nowhere, but at least I'm
learning something. And so that understanding helped
me to kind of not kill myself doing that.
So I think it's it's striking that balance.
I received the question along the lines of system thinking
earlier as well and I I'm not sure how I would explain to
people how to start thinking in systems.
I'm curious how how you would describe it since it kind of
feels from your perspective you had it in education.
Yeah, Yeah, that's a really good question because I read the book
The Donna Meadows Thinking Systems and it starts with the
bathtub and and goes from there. But I don't think I it actually
developed my thinking. I guess I don't know how.
How do you start? I guess, I guess you just, it's
practice, like you start looking at problems and rather than
thinking, I guess you start taking the individual out of it,
especially in in human challenges.
I mean, bathtubs are one thing, but you know, most of the things
engineers are dealing with are human organisations.
And if you start taking, if you start thinking empathetically,
like why is this security engineer saying no to me all the
time? It's psychologically easier to
think they're just, they're just an idiot.
They're just, they're just, they don't like me, you know?
It's an easy out. It's an easy out for every for
you, but if you if you dig into it then you you can you can
understand it better. But I don't really have a better
answer than that. Yeah, I do feel like solving
problems and figuring out where I'm battling systems versus what
is then causing an issue. I think that really helped.
But you and I both worked in operations.
The default example that I have in mind stems from my experience
in operations where my colleagues were doing the same
thing over and over again. And I just started and I was
like, there's no way in hell I'm going to be doing this thing for
the rest of my life. That was my thought early in
career. And then I, I talked to some
people, it was a change needed in the database.
And then those symptoms never happened again.
And that for me was the most concrete example that I could
think of. But it has a lot to do with,
first of all, patients, but second of all, so a little bit
of curiosity, why are things happening the way they are?
And like you mentioned, empathy plays a part in that when we're
talking about other people also acting in a certain way.
But I find this question quite particularly hard.
Yeah. So I mean that that whole thing
of, of why do some people solve the root cause, find a way to
solve the root cause and why do some just happily, I mean, some
people, you know, it made me think actually, I got into
Erlang quite, quite heavily when I was working at that, that
gambling company because it was really good a fit for, for some
problems we solved. And I spent months teaching
myself Erlang. I built like a little poker
server in it. And yeah, it was great.
And we, we started writing a tool for real time updates of,
of price changes and Erlang's a perfect fit for that.
So, so we built that and we said like we should, we should be
doing this, you know, we should roll this out across the
company. And people in senior position
were like, I'm not sure about this.
And I just thought they just don't get it.
They don't get it. Erlang is that's the way.
And we, they said, OK, we'll try it with, with an engineer on a
project. And they picked a kind of
typical engineer and he really struggled, really struggled with
it. And like he was motivated and he
wasn't stupid at all, but he's very competent.
But he just wasn't into it and. Then it's a hard sell.
It's yeah, he wasn't into it. And and it was like, well,
that's the litmus test. You have an, you have 700,
basically 700 engineers working for this company.
They're not going to all learn Erlang.
So either this is like an elite thing that only three or four
engineers are going to do and that's one, that's one option.
But I didn't really think about that.
It was only afterwards I realised, oh, it did fail and
that's why it failed. And actually it wasn't about the
technology, it was about the adoption.
We had similar challenge. The reason I got into DACA was
because we didn't have any test environments.
We didn't have any really any CI, which sounds shocking now,
but we just didn't. I know it's different now, but
Docker allowed us allowed me to put together all these scripts
and just shove it into a single container and then daily builds
do a daily build of each customer's environment.
Yeah, perfect. Yeah, it was, It was brilliant.
And I remember getting up at a conference and saying people
were saying like, why don't you just, why don't you just use
Chef? And I was like, we have 600
engineers, you know, teaching them all chef is a big deal.
And then the guy got up after me and was like, I don't get why
teaching 600 people chef is a. Skill issue, yeah.
Yeah, it's like a skill issue. And we were like, OK, yeah,
fine. But actually that was that talk
where the guy kind of put me down.
That talk is where I got the job at the bank because the guy at
the bank said this guy understands.
Knows what he's doing. Well, this guy understands that
it's not just about the tech, it's about all these other
things. But he can do the tech too.
So let's let's go talk to him. Yeah, incredibly important.
I love that in the moment you have these points and I also
have these where you might not understand where certain
decisions are being made, but I feel like as you get more
experience, you appreciate those moments because they taught you
something that you didn't realise at the time, but now
reflecting, you actually understand them better.
It shows a lot of growth I think, and I wish that upon a
lot of people. I feel like if you experience a
lot of things and you have a way that is like reflecting.
For me, it's a lot of these dialogues to be honest, because
I love picking people's brains. I love reflecting based on what
someone says, also on my own experiences, and I'm having a
lot of fun doing so. But through that I learn a lot
about my own journey in that way, and I hope people have
environments and also people where they can do the same and
kind of brainstorm in that way. Yeah.
I mean, one of the changes I've got now, so as I mentioned, I'm
CTO of this consultancy is I know I can see across my
organization, we've got these engineers who are having these
amazing experiences, like really interesting experiences, but at
the same time they're under pressure because, because when
you're on a client, the client kind of consumes you.
You're, you're really just thinking about the client.
But I want our engineers to learn from each other.
And I've, I've struggled with this so much like how can I get
this cross pollination and this cross, you know, this experience
to crossover. And the best way I've found
actually is having a Friday lunchtime.
Let's just have a chat about what's.
Going on, put them together. Yeah.
And I, I, I'd say like you're not allowed to prepare if, if
you're going to talk on something.
Well, no, you don't have to prepare.
Like if you come unprepared, that's totally fine.
And that allows the people who are focused on the client to
just turn up and just start talking.
And we fill the hour every time. I always think like we haven't
got enough to fill this hour, but people then start talking
about this, then the other. There's a lot of discussion of
AI at the moment. Of course.
We also have non-technical people coming along and we're
having to explain to them how AI works and they're super
interested and we're trying to we're trying to communicate with
them. And I think just that whole
that, that forum, it's just a little bit like what you're
talking about, I think is a is a wonderful way of improving the
experience and value of our, of our engineers.
Because one thing I love about consulting is you're you're
meeting all these different clients with different content.
You can learn so much faster. Yeah, yeah.
If you like finding people around you that are very
enthusiastic, that have this passion and they're also open
and have the patience and willingness to share, that's the
best environment to be in. Those are the environments you
want to seek, cultivate. I, I take hiring very, I mean,
you have to take hiring very seriously if you're, if you're
running a consultancy. And I always tell the people
when we're hiring them, like what are we looking for?
We're looking for curiosity, a certain level of technical
capability and adaptability and ability to communicate.
Those are the things we're really looking for.
And you know the the technical capability is a as a kind of non
negotiable. That's what gets you in the.
Board, but do you need, do you need to know everything about
everything? No, you just if, if you're
curious and you're motivated and you can communicate well, you
can go very far as a consultant. So yeah, that's and, and so it
comes through in these, in these sessions we have because
everyone is like super, super interested in sharing their,
their knowledge and, and, and experience and learning from
each other. That's really cool.
Yeah. I've really enjoyed this
conversation. Me too.
Yeah. It was really fun figuring out
what makes seniors go beyond that ceiling, I would say.
And then also picking your brain about certain systems and
systemic thinking. That was a blast.
Thank you so much for coming on. Thank you.
Cool. We'll round it off here.
If you're still with us, let me know in the comments section
what you thought of this episode and we'll see you in the next
one.