Thank you for listening and supporting the tekhelet journal
podcast. It is my ultimate pleasure to
serve you in the past two years. I hope the podcast serves you
well and gives a lot of insights that you can apply in your work
and Life as we are going to celebrate the 100th episode.
I would appreciate if you can drop a few words in our survey
related to your experience with the podcast.
And if you record your story or message in an audio format, we
may feature you in the podcast special episode.
The survey is open for both. Both listeners and guess.
I'm looking forward to hearing your message.
Soon, drop me the message on technology, know the dev slash
celebrate Dash 100, here's the link one more time, technology?
Know the Beth's, let's celebrate bash, 100, what engineering
means? You know, the disciplines is the
most effective efficient way of doing work of high quality.
And so, if software engineering, doesn't allow us to build back
to software faster, it isn't software engineering.
Hey everyone. My name is Henry Surya, we
Robin. And you're listening to the
technology, you know, podcast the show where I'll be bringing
you the greatest technical leaders practitioners and
thought leaders in the industry to discuss about their Journey
ideas and practices that we all can learn and apply to build a
highly performing technical team and to make an impact in your
personal work. So let's dive into our Journal.
Hello to all of you, my friends and listeners, welcome to this
special episode of the technology on our podcast, the
show where you can learn about technical leadership and
Excellence from my conversations with great thought, leaders in
the tech industry and this is the episode number 100.
Oh my God, I can't believe that. I finally reach this special
milestone in the beginning. When I started my journey
creating this podcast, I made a pledge to myself that I would
reach 100 episodes. And here we are two years later
from my very first episode. I'm really proud to have reached
this Milestone and I could not have done it without the support
from each of you, who listen to the episodes week in week out.
And also a special thanks to all the guests, who appeared on the
show and for sharing your expertise and wisdom.
I really, really appreciate your trust collaboration and for your
willingness to share back to the community.
And personally, I have grown a lot.
In the past two years by serving, all these great
contents. And I really hope that all of
you enjoy and grow yourself as well.
And by the way, if this is your first time, listening to tackle
e-journal, make sure you subscribe.
And follow the show on your favorite podcast app and the
social media on LinkedIn, Twitter and Instagram.
And for any of you who wants to contribute back to the creation
of this podcast, support my journey by subscribing as a
patron, at technology node Dev slash pay.
Patron for today's special episode, I am super excited to
share my conversation with another great thought leader in
the software industry, since I read his groundbreaking book,
continuous delivery, and applied.
Many of the principles and techniques mentioned in the
book. I have benefited a lot in my
professional career and still continue to Advocate its best
practices wherever I can. The book itself, one the joke
award in 2011 and has been continuously impacting the
industries. Instant even until now.
And so, as many of you would have already guessed by.
Now, my guest, for today's special episode is Dave Folly
apart from co-authoring, continuous delivery book, these
days. Dave, also runs a highly
successful and popular continuous delivery YouTube
channel on software engineering topics, which I also watch and
learn from it. A lot in this episode, we
discussed Dave's latest book, modern software engineering,
Dave started, It by explaining his view on Modern software
engineering and why it emphasizes on practices for
building better software faster. Dave describe the foundations of
the software engineering discipline and he explained the
core competencies, we need to succeed in modern software.
Engineering by becoming experts at both learning and managing
complexity. They've also explained the
importance of understanding technology, fundamentals,
improving software, readability, and handling software.
Cassidy by managing concurrency, and coupling towards the N Dave
also share some of the tools in the modern software engineering
tool kit that, of course, include continuous delivery.
I really enjoyed my conversation with Dave learning the
fundamentals of modern software engineering.
And importantly, the practices that we can use for optimizing
learning and managing complexity.
If you also enjoy listening to this episode, please share it
with your friends and colleagues, who you think may
also benefit fit from listening to this episode.
It is my ultimate mission to spread this podcast to more
people. And I need your help to support
me towards fulfilling my mission.
Before we continue the conversation, let's hear some
words from our sponsor definitey is the top International
software development conference, with an emphasis on coding
architecture and Tech leadership skills.
The lineup for this year is truly stellar and features many
Legends in software development names, such as Robert Uncle Bob
Martin, Kennebec Scott Hanselman, thank a subramanyam
Carolyn honey, Alan Halep, Mary, poppendieck, and many other
prominent names, including some of those who have also appeared
in this podcast before the conference takes place online.
So you can enjoy it from the comfort of your couch.
We spoke to the definitey organizers, and I'm happy to
share that technology. You know, has got the 10%
discount code for you, enter the promo code, awsm underscore tlj.
When you purchase the ticket on definite e.com, here's the promo
code. One more time awsm underscore,
tlj. Depending on the time when you
purchase a ticket early price is still available.
See you there. Today's episode is proudly
sponsored by skills matter. The global community and events
platform. With more than 100,000 software
professionals here members. Can organize their learning
experiences around the technology topics.
They care about. At most you get on-demand access
to their latest content thought, leadership insights, as well as
the exciting schedule of tech events running across all time
zones. So whether devops our data
science is your bus or you are fan of functional programming or
all things Cloud, you can make real connections with people who
share your interests head on over to skills method or calm to
become part of the tech community that matters most to
you. It's free to join and you will
find it easy. Keep up with the latest tech
Trends. Hello everyone.
Welcome back to another new episode of the technology on our
podcast today, I'm very excited. So I got the chance to meet
someone who I follow a lot by reading his book, following his
blocks. And also recently his YouTube
channel, his name is David Farley.
Also more widely known as de Farley.
I mean just to share. And when I read The Continuous
delivery book, this was like many years back.
I find that book is one of the Cornerstone of my career.
Trying to implement those practices like delivery pipe
lines, continuous integration. I really find the book is
something that is really critical for my journey so I
really thank you for that for writing the book and as a next
artwork as well I always love to see all the other workers as an
author because they write a very insightful book based on
experience. Thank you so much for spending
your time today. Dave looking forward for this
conversation It's a pleasure. Thank you very much for asking
me and I'm pleased that you enjoyed my book.
So in the beginning, maybe if you can share a little bit of
background about yourself, maybe telling us your career Journey
that maybe people haven't heard about any highlights or turning
points, should I do? So, as you can probably tell by
looking at me, I've had quite a long career, I'd be doing
software developed for a very long time and Correll.
I started off probably at the start of what was then called
the microcomputer Revolution. So the Mooshroom kind of Lady
computers to computers that were more accessible based on Silicon
integrated circuits that were starting to become cheap.
So I had the advantage of kind of the first computers being
fairly simple and be kind of growing up with them over time,
which is pretty nice. Really, I've done quite a lot of
different things. I spent the early part of my
career with Hardware manufacturers, I worked for
computer manufacturers and writing operating systems and
device drivers and those kinds of things.
And then later, I got interested, I don't know by
accident. I didn't but in kind of
distributed systems starting to build for those days larger
scale distributed systems, I think I ever shared for kind of
become part of the team that invented the concept of
microservices a long time before they actually cook off.
So we kind of emailed them and then they went away again.
We did some really interesting stuff for a research company.
Building software with these very Loosely coupled.
We called the collaborative business objects.
It's still the most interesting software in the sense that you
could introduce two pieces of Independently developed
software, and they could do useful work together, which are
not seen, anything do that, quite, as well.
It was a fascinating project to work.
On one of the seminal moments, for me was, as you mentioned
Working For Thought Works, weren't you thought works, and
I'd already be doing informally some agile.
Things I'd be doing a little bit extreme programming projects
before that but thought Works kind of turn the volume up on
that show me and we started doing what we thought at the
time. And I think the one most people
credit to Sweet are working on some of the Big East agile
projects that had been done up to that point.
And so we must are pushing the boundaries and finding
difficult, things difficult problems.
We've had you, apply extreme programming.
When you got 200 programmers, we started addressing that kind of
thing. And I think that was one of the
things that sort of gave birth to the concepts that have
steered my career. Since in terms of continuous
delivery and talk to you more engineering, kind of approach to
thinking about solving, these kind of problems.
The latter part of my career. I've spent nearly all of my
career building real software for people one way or Nova about
six years ago, my friends and family convinced me that I ought
to start my own business because I was always a bit nervous of
doing that kind of thing. And so I started a consultancy
these days. My job really is a software
consultant advising organizations all over the
world. Usually big organizations on how
to do a better job of software development and not to mention
also producing YouTube content. It gasps you seem to be very,
yeah, even of churning out contents every week.
Sorry to Oh it's you. That was some accidents as well
as you can probably tell. I'm not some weird plants are
career hoop. That was an accident forced on
us by the tender me. I was very busy with
consultancy. The pandemic happened and that
stopped and my wife and I were looking around saying, what are
we gonna do? We especially go to Brazil next
week, you know, probably gonna be there.
And so we started YouTube channel, partly just as a means
of keeping us sane and having something to do, and it's being
extraordinarily more successful than we ever anticipated or hope
which is Not. I can also see the plaque behind
you, so I think that, yeah, I know like after you reach some
hundred thousand subscribers or something like that.
Right. That's right.
So we got that earlier this year, we just passed a hundred
twenty thousand subscribers we rapidly approaching or we just
crossed the 5 million views Ma. So it's quite a big Channel,
know for a technical Channel. We release a video every week on
Wednesday and we get tens of thousands of years.
Every week we usually would get that quarter million views a
month. That's quite a lot of people
coming along with the journey and at least listening to these
ideas even if they don't always agree with them, which is good
because we're back at the debates and it's not just about
the number of viewers, but I think the quality of the content
at these pretty rare still, I rarely find YouTube videos that
covers all this technical contents, that is deep and also
based on real experience. So, thank you so much for
producing that and I hope that it will continue to grow.
But today, I think we will discuss mainly from your latest
book, which is titled modern software engineering.
The first thing that comes Two minor for me.
When I look at the title, why do you call it modern software
engineering? Is there the always of software
engineering and there's a new modern way of software
engineering? Yeah, it was a total movie stuck
in my mind for a long time while I was writing the book and to be
honest I was somewhat nervous. Like the reason that I call it
modern Subterranean rather than just software engineering and I
was nervous. Oblique lightly is kind of the
same. I think, you know industry, we
kind of weirdly devalued, what we think of as engineering I
It's very common to think for people to say, software
development isn't engineering and a lot of it isn't nothing.
That's fair. And lot of it isn't engineering,
but in other disciplines engineering, doesn't mean
heavyweight bureaucracy, which I think is what it tends to mean
to us. What engineering means in other
disciplines is the most effective efficient way of doing
work of high quality. And so, if software engineering,
doesn't allow us to build better software faster, it isn't
software engineering. So modern software engineering,
I think he's different to what we're before we tried to
implement engineering approaches before and they were nearly all
works. Narrowly focused on some
particular set of tools or diagramming techniques or
something like that and bureaucratic in heavyweight.
And that's not what I mean at all.
So, I mean almost entirely the opposite of that.
So what I was interested in part of my background and interests,
my hobbies is that I'm very interested.
Science. I think that a reasonable
definition of engineering, is as a practical implementation of
science. You apply scientific style
reasoning, which is Humanity's best approach to learning to
solving practical problems. And I think that's a reasonable
kind of colloquial. Definition of engineering, I was
always interested in those kinds of ideas.
What are the ID that we could point to and whatever your
technology, whatever? The problem, you're solving,
whatever organization that your work?
If you applied these principles, you'd have a better chance of
success. And I thought I'd spotted some
of those things over time, and then related to continuous
delivery, which was my previous book that I worked on with Jay.
So humble and so on, but also somewhat separately.
But slightly different angle. It's kind of all fog.
They intersect and they're very reinforcing at one another but
they're not the same idea. So what were the principles that
would allow us to do a better job?
And I thought that was what I was trying to explore and that's
what Titles meant to try and convey the subtitle of the book
itself is doing what works to build better software faster.
And in your book, I also find it quite insightful.
Is that you mentioned that if our software development
practices do not allow us to build better software faster, we
should really change them because they are not enduring.
So please tell us more about this concept.
Why do you think so like why we should focus on building better
software faster? The simple answer is, why would
anybody want to build worse software slower?
I keep this the combination in part.
This is my kind of again colloquial take on the Dora
metrics, the metrics that come out of the state of debauch
report that give Alice a predictive model.
If you work in these kinds of ways and do these kinds of
things, you increase your chances of success and they are
based on the stability metrics, what's the quality of our
software? When we eat You suckering to
production, how often does he break and when it breaks, how
long does it take us to recover from the breakage and fruitful,
what's the efficiency with which we can deliver software of that
quality? So that's the fast apart.
So working quickly, one of the findings from the door and
metrics which is important, I think, is that there is no
trade-off between speed and quality.
One of the traditional assumptions is that if you want
to build high-quality things, you need to go slowly what
Madonna reports Set is know, if you'd go slowly, you build
lower-quality things. And that's what the statistics
say. That's what a sociological
approach to measuring the performance of subdued.
Lot of teams finds the more barriers you put into being able
to release change quickly, the worse the output.
So we want to be able to move quickly, we want to be able to
go faster and we want to be able to build better software, and
that means that we need to be able to focusing on be able to
learn find out where mistakes are correct and really quickly
and efficient. And that color is, one of the
way. Which is closing with continuous
delivery. Can you still use a fantastic
way of being able to build better software fast from my
career itself? I can tell that so many
experience from my view is that we build software faster in the
beginning. But since that, after a long
period of time, or maybe few months, it starts to get slower
and slower and maybe many people have left.
They don't know about the context that domain problems.
So I think maybe some of the tools that we will discuss today
can help people to really build better software faster like
continuously. So, You have a definition about
software engineering, right? If I can read here, is that the
application of empirical scientific approach, which you
mentioned in the beginning, just now to finding efficient
economic Solutions. So, the economic Solutions here
is also quite interesting to practical problems in software.
I know that you will cover in the book that some of these
keywords are really important. Maybe if you can discuss
slightly some of these terms, why do you think it's important
to Define software engineering using this way?
So, I was trying to get the simplest minimal description of
what I was thinking. In terms of engineering.
I was quite pleased with that description that you just read
out because it's fairly lean. It's fairly concise, but I think
he doesn't miss anything important.
So applying the reasoning, the rational approach of science, I
think that one of the fundamental Nature's of science
is that the difference is between science and almost.
Everything else is that we start out assuming that we're probably
wrong and that's a very healthy thing to do.
In engineering and in software development at all.
So we're going to apply scientific style of being in a
lightweight, not overburdening, not bureaucratic way.
But we're going to be informed by that kind of thing.
We're requiring. That solving practical problems.
We're not doing theoretical researcher.
Proving the quarks are the basis of particles or anything.
We're building things. And so we've got to be practical
and pragmatic. So the empiricism of engineering
just trying stuff out, ultimately, he's a big part of
it as well. The economic power is Again,
part of engineering. If we are physicists, we can
imagine building machines at a black hole, thinking about that
practically economically in energy terms.
We're not ready to do that kind of thing for us Engineers, we
are building things, and so there are always economically
strength at some level. That's about the amount of time
and effort and money that they spent, but also economic
constraints in the sense of the performance of our delivery
system and the performance. It's about software.
The carbon footprint of the software that we create, that
kind of thing. He's an interesting question for
you need. So right.
So I wanted to get that in there as well.
So I think all of these different parts of the
definition, kind of lead us in this direction of trying to
organize our thinking in a more rational way and apply it to
solving problems, and are of value to people in some way.
That's what I was trying to reach full with the definition.
Thanks for that explanation about the software engineering.
So I think for us, we need to probably redefine some of our
understandings of engineering is not just writing code that works
and deployed. I think that's a really good
point. One of the things that are
obsessed with at the moment. He's watching SpaceX built a
space Rockets to go to Mars. That's engineering Live on
YouTube, you can kind of watch it happening in the real world
that's not theoretical. They're evaluating their ideas,
their blowing spaceships off are learning from that.
They're making progress. Final step and of the old of
these things, kind of coming to an engineering approach to
solving problems and thinking about the ways in which stuff
can go wrong in all of the time, looking at how to be proved and
refine our designs and our Solutions, so that they'll
better fit, the problem that we're trying to try to solve.
For us to follow this definition of stuff, engineering you
mentioned there are two things that we have to be expert at.
So one is actually to become expert at learning.
This comes back. Probably to the scientific
approach that you mentioned. If s is experts at managing
complexity, maybe we can cover one by one.
Let's start with experts at learning.
So you mentioned there are a few key things like five things that
we need to be able to practice in our software engineering or
software development day to day. So what are those five things?
Yes. So experts at learning, let's
just say why that's important. First of all, it's to do with
this kind of scientific engineering philosophy of
assuming that we probably run, we not going to start out with
perfect vision. We're not going to be able to
predict the business context, the technical context, the
socio-technical context of how we organize ourselves to what,
we don't know, the results, and you've those things at some
level, we're going to get all of those things wrong.
And so we want the ability to make progress while we don't
know at the start of a project when we don't have all of the
answers yet. Therefore we want to optimize to
be able to learn Microsoft did some research into their own
ideas and found that two-thirds of their ideas produces zero or
negative value for the company. So you'd like to be able to
learn which with a bad ideas early on, if you wanted to be
efficient. So we want to optimize for be
able to learn all sorts of things.
What the problem is that we're really trying to solve what our
users really need to be able to solve that problem.
What a technical solution, Speech.
Does it really work? Is it releasable?
Is it compliant all those things?
You will learn ways of doing that and we going to work in
order to be able to do that. As you said, there are five
principles that underpin our ability to do that.
So, we need to work intuitively. We need to working many small
steps so that we have many opportunities to kind of look at
what it is that we doing and refine it.
Oh, that didn't work. Let's not do that anymore.
That kind of thing. We need to take feedback
seriously. We need to understand what
information. Animation.
We need to gather in order to be able to determine whether our
ideas or our Solutions are useful for help whether our
problems are landing with our customers, where there are tests
are passing or failing, we want to optimize great feedback to do
that. We want to also be able to work
incrementally, want to be able to build systems or almost
evolutionarily, sort of bit by bit knots.
All in one, great, big chunk. So, we want to be able to deal
with these things more independently, so that we can
kind of carve it out in certain if we working on it.
Scale divide the work so that different parts of the
organization can work on different parts of the problem.
So working incrementally is part of all of that.
We want to book. Experimentally.
If you want to start being more scientific, be more like
engineers in our approach. Then we should apply a little
bit of discipline around the way in which we think about making
changes. And the key to that for me is to
think experimental e. We want to make little
predictions. You want to say?
Well this is the problem that We're looking at the moment.
We think if we did this thing this would improve that position
on this problem. How would we tell?
Whether that thing was or wasn't working?
That's the difference between an experiment into that step of
predicting, what the outcome is, and then carrying out the tests.
So you can think of this as a etched with the product.
We think that if we do this thing, it's going to improve the
sales or this widget and we'll put it into production, see what
happens or you? Could be as a trash.
All. I think that the coach should do
this thing. If it were to do this thing,
this is the assertion that I would be able to make.
Then I write some code to fulfill that assertion, these
are ways of it working experimental e and that can.
And for me, has become pervasive.
That's the way that I now think about and practice nearly all
change where I can. So what is it that I'm trying to
do? How would I understand the
results of this thing? And I'm just thinking in that
way, doesn't need to be heavyweight to be really Fast
really short, really efficient but it's just a slightly more
disciplined organized. Way of structuring, I think that
gives us better results and the last one is the empirical we
already talked about. You'd be racism.
Engineering is about real-world things.
You cannot about some Ivory Tower imagination of a system.
It's about the reality of it. So we need to gather feedback
from production. We need to go to understand
what's really going on. We need to be able to make
practical decisions based on what it is that we find Elon
Musk. He's building space.
Citing checklist. I'm sure that they're doing very
sophisticated, computer, simulation, but at some point,
it builds the pads of metal and that they blow it up on a test
and see how it works. That's how Engineers think, and
that's how they operate. If you look at the development
of any sophisticated engine, think about a layer of plain
error. Planes used to be incredibly
dangerous things in 2017. The equivalent of two, thirds of
the population of the planet flew in commercial airliners and
not one person died that wasn't possible in the early version of
Severa planes. We had to go through that pain
of things breaking and killing people and say, oh well, we
won't do that anymore. Will relearn some eight is about
work incrementally and growing and learning based on mistakes
that we make fundamentally. And we've got to give ourselves
the freedom to be able to operate that way and to make
those kinds of mistakes books when they occur to learn from
them and to correct that. So I think those five things
it's true. Working feedback, incrementalism
experimentation and empiricism are the Cornerstone of our
ability, previous species to be up to learn, effectively and
efficiently. I can't imagine anything that we
could consider to be engineering with that post legs.
So the first third of the book is really about exploring.
Those ideas in much more detail in looking at each one of those
principles and Hunt picking apart and seeing how it fits
with software development. When I read these five key
principles, they seem to interrelate with each other.
Of course, they support each other and then you become expert
at learning. So these days, many people, I
think already understand a how they work intuitively.
They probably tried to build a shop feedback loop and things
like that. Where does that thing go wrong?
Why do you think still you still need to reiterate these kind of
Concepts because a jolly five understand correctly.
From the beginning, from the get-go is actually to adopt all
these kind of things, right? It right.
If incremental short feedback loop and there's also an
experimentation partway after the spring, maybe iteration that
you deploy and learn from the customers.
So, why it seems like these kind of things are still forgotten or
maybe misunderstood, I've Had The Good Fortune to you during
the course of my career to be fairly close to the birth of
some influential, Big Ideas, my observation in all of those
circumstances. If you talk to the people who
came up with the big Ideas after their ideas are successful, they
said we didn't meet, man. That's because this kind of this
semantic. Diffusion things people come up
with ideas and then other people interpret them differently.
And I think that's definitely happened to Agile.
I think if you look at the agile, Manifesto, which is the
closest thing to a definition of what agile really means,
scrubbies near the agile, Manifesto says, what agile is
really about. If you look at that that's not
going to a day that's still pretty good.
That's a pretty good specification of what goes on
the problem with a Maturity as an approach, an armored car
carrying a job. A believer that is an important
step. But the problem with it is that
part of it is about giving people freedom to make their own
choices and learn things and that's deeply important.
But there are naive ways of taking that and one of the naive
interpretations of that is that everybody has a veto on
everything. So, if you're working on a team
and your team decides that they're going to be testing
development, and one person on the team decides, they don't
want to do testing development. Don't they won't do it.
They'll Victory. That's not really what agile was
the level at which autonomy works.
I would say is kind of at the level of the team.
So, team agree begin to work this way.
And then you do it, whether you like it or not, you do it up
that be, they're part of the team that was doing things that
I didn't speak with the right things to do.
But we still did it as a team because that's what we'd agreed
to do. So, there are some things like
that. There's also the natural
resistance of a big organizations to want to change,
but I think my purpose, Over the years, I've worked with a lot of
different organizations usually bigger organizations but a lot
of I think my overall summary if I could love for that it's very
unfair to lock the Bots. Get the body Force above them
all together. If I was to put words in their
mouths, what they all want, is that we'd like to carry our work
new. Tack means we always did before
or get different results and that's not gonna happen.
So, the trouble with agile ways of working is that they're quite
radically challenge to traditional corporate culture.
So they're very difficult for big organizations to adopt
because it means breaking down barriers in organizations
without sounding too immodest. I think the continuous delivery
is currently state-of-the-art software developed eat what most
of the best companies in the world at software development to
do and it works, you got data that says that it works often in
certain measures orders of magnitude better than other
words of doing. And I think the one of the
advantages of continuous delivery is, it's just a little
bit. Prescriptive it still gives
people the freedom to innovate but it's fairly hard line on
some things. So part of the definition I use
these. If you can't produce something
releasable every day, it's not good.
English delivery that's pretty hard.
Why? That's a pretty definite State
that's more definite and say as important as it is but people
over process which the agile Manifesto says he's a really
important idea but it's a bit vague compared to produce
software of the releasable software.
Every day. That's a pretty hard line
statement, a little bit more constraint, a little bit more
specificity in what we should be doing.
I think sometimes freeze us and helps us to be more Innovative
to be more creative in some of the things that you do.
I think a continuous delivery of the second generation agile,
discipline. I think the second generation a
job approaches are more effective than the first
generation wants. So are continuously abuse.
Also second generation extreme programming for me was the best
agile. This Knights of the first
generation, but continuous delivery adds a little bit more
stuff around. Extreme programming takes a
little bit further, but both of those are engineering
approaches. And that's the other thing.
That is a failure of agile, agile, got sober T to become a
project management approach rather than an engineering
discipline. Ultimately, we're building
software. So we got to be able to create
things. However, good, you're planning,
he security doting, the stuff, it's not working at So I kind of
like the our definition of continuous Delirious like a
second generation of a gel or maybe experience.
Because some of these practices is like, a rigorous process.
You have Department pipelines, running tests as driven
development, and all that, I think they're kind of, like,
just people guidance are so, what practices that they need to
follow. You mention that if you cannot
really software every day. So it's a really high bar,
especially for people who started with a good idea in
mine, and it seems like very far-fetched, very high and Big
Ideas. How can people do this while at
the same time? I think one of the discussion
point in the book you mentioned about cause of change.
So if you're doing traditional software development, maybe it
gets faster in the beginning, but over the time it gets
slower. But we're using this modern
software engineering approach. You can actually make the flat
course of change to the software developers.
So tell us more how we can actually aspire to reach that
level where over the time. Basically any changes you make
to the software is kind of like flat.
It's not like a steep curve going upwards.
I think actually that brings in the kind of the second section
of the book. Which is what I call managing
complexity, the techniques of managing complexity are really
about addressing that problem. So I think that one of the most
important qualifying characteristic of what, we would
think of as quality in software, is our ability to change it.
So sure it has to do the job that it was designed and
intended. Rory, if performance is an
issue, we like fast enough at all those kinds of benefits.
But fundamentally it's living in this world where we don't know
all of the answers on day one. We can't perfectly predict the
future and the business Direction.
Then inevitably, we must be able to change minds and change our
software. The business Direction might
change. That solution might not be good
enough in some way or another or we spot a new idea.
That's much better than the one you had in the first person we
want to follow that. All of these are reasons to
change our software. So I think that one of the Of
the cornerstones of high quality is our ability to make those
changes quickly fishing, the easily flattening that cost of
change curve. As you mentioned for that, we
need to be able to work a little bit defensively, the way in
which we approach designer software.
So it's reinforced by the practices of learn, if we're
building our software so that we can iterate quickly on it and we
can gather good feedback from it and we can carry a little
experiment to change to it as a licking experiment of some form
that reinforces this ability. Then we need to build software,
that's modular that's cohesive. The bits that are separate
should be separate in the software, the bits that are
closely related should be close together in the summer.
We want it to had good separation of concern, each
piece of that software to be focused on doing one part of the
job. And then when we need to change
their mind on that part of the job, we can just work in that
part software. We want our software to lines of
abstraction wanted to hide information, between one part,
At the software in another so we can make change in one place.
So that worry too much about our other places work.
Having a coupling between the software is a problem and
abstraction will give us that. And that is of the last point,
which is to manage the coupling effectively, but with at
registrants attendance, eat more in the direction bluescope.
So, we'd like, at the pieces of our software, not to care too
much about the detail of all the pieces, if we build software
that looks like there, then it is much easier to change if you
want to build software like that.
Then one of the It makes the beginning news that massively
amplifier is all of those five properties test-driven
development, prefers software, that's modular.
Cohesive separation of concerned abstract and Loosely coupled,
it's much easier to write tests, urine, tests or software, that
looks like that if you have those properties in the code.
So, we got this nice feedback loop, this nice reinforcing
process. We want these properties as
Hallmarks and quality in our code.
And we have these techniques that tend to make us put these
properties in. The code that's kind of what I
mean by engineering. It's not an automatic recipe.
It's not going to automatically generate great software, you
still need to be good, you still need to be intelligent, still be
to be thoughtful. But when you are good
intelligent and thoughtful and use these techniques, it
guarantees you a higher probability of a better outcome.
It's not going to make sure that you're going to be successful.
But you've got a higher probability, you will be if you
work those ways but big what the data says and what I'm trying to
do Book. So when you mention all these
five key principles, cohesion modularity, loose coupling
suppression of concern, looking back in the University, we kinda
like, thought about this maybe in programming 101 or things
like that, but it's kind of funny.
Like, why do we need to keep on repeating these sort of
fundamental basic Styles in software engineering, probably
or computer science? You mentioned the industry
currently has a tendency to focus a lot more about tools
Technologies Frameworks or languages rather than actually
emphasizing a lot of fine. So tell us how do we get wrong
here? So many people talking about
Technologies like Cloud, kubernetes mobile, whatever that
is, but why people the emphasize on design?
It's a good question and I wish I knew the real answer I can
give you my opinions as to some of the Rings.
I think three degrees, inevitable software development
is a career appeals to certain kinds of people.
Those of us that it does appeal to our interesting, how
computers work in the tools. I'm as likely as anybody else to
have a bigger wrist debate about the merits of different
programming languages in a pub with anybody else.
But ultimately doesn't really matter.
Very much collapsed language where I thought that I'd kind of
consciously Works to use every last part of the language with
see many years ago. There's not many parts to sing
so you can see on the bearer of small brand.
But as part of the book, I tried to count up language like used
in professional settings over the course of my career.
I reckon I'll probably Programmed commercial software,
one way or another in about 20 different languages.
So very ephemeral over the course of 40 years.
That's one language every couple of years.
So you might be more stable than me.
You might be looking at that more.
I've certainly done more stuff in Java than I have here
prologue. But it is interesting thinking
about the wet job works. If currencies in syntax in Java
and python a lot because that, you talked about nothing but
fundamentally when I stop and think about it doesn't have any
impact. From the way that I design
software area, it has some new arts in terms of whether I
medium Attic in the language that I'm using or not.
But fundamentally the way that I organized and carve up the
problems that are trying to solve with software hasn't
changed very much for decades in some ways we find around the
edges. The way that I think about is I
think a little bit better now but the same things that were
important doing things in C or even an assembler are still
importance during the in python or JavaScript so F sharp or
whatever else. Those Fundamentals, don't change
modularity, Max's whatever the technology that you're using to
buy Cody's at the same separation.
Concerns the same. As you said, we learned about
these things in University and then we discard the nurse at
age. Why do this with Helm charts?
That stuff? Doesn't really better as a
result what we tend to do. So we look at something like
kubernetes, kubernetes is a fantastic tool.
If you're Google scale, doing all of these different Services.
That's fantastic. I see teams.
All of the time that rebuilding Forgive me some very simple
little application. That's probably a single process
running somewhere and they start off by deploying you with
kubernetes you going. Why does kubernetes helpful,
this simple, little problem and is that work?
So thinking, in terms of what the problem is, that we need to
solve applying things like modularity.
Cohesion, separation of concerns tells us how to solve it and
then we figure out the tools to use to apply to maximize our
ability to build multiple software, Oh, that's so easy
with good separation concern. That's the goal.
The technology is just, the means like a carpenter being
obsessed with the nails that they use and the Hammers rather
than what it is that they're building.
We need to focus on what it is that we're building.
Not the nails and the hammer thanks for giving your insights,
like you mentioned, any good programmers, doesn't matter what
language maybe they can learn new language or even use old
language. They can still produce software
that resembles all these five key elements.
Verity separation of concerns and things like that, the
importance of managing complexity at resonate with this
as well. Because the more you have
growing complexity codebase the cost of maintaining that
software is really huge and we are probably wouldn't be able to
make change as easy as maybe the first time it was written in the
last few episodes of the podcast.
I've talked with some of the guests about managing code,
complexity, readability, and all that.
I think this kind of ties back to all these principles of
design. So why do you think it's very
important to have code that is manageable in terms of
complexity and also readable you mentioned is about
communication, but this is counter-intuitive to people
because people think writing surface is to make the program
work or to solve a business problem here, and that's
certainly true, but it's never one up.
So this sustaining, our ability to solve those problems, he is
in everybody's interest, it's in the interest of the
organisations that employers as well.
Run. And I think that's one of the
mistakes that as software developers on teams.
We tend to be too short term in I think and we tend to optimize
of next week rather than next month if you optimize for next
month that some different things it's a bit like deciding that
the time that you spent putting Petrol in your car or charging
your electric car or is a waste of time.
And so you're going to do with that and very quickly, you're
going to run out of gas and you let your car will come grinding
to a halt. If you do that thing, there are
sir. Certain things that you need to
do in order to maintain your ability to make progress in
software terms. That's precisely what you said
at the start, which is the big problem of building complex
software is that over time it gets harder and harder to change
it. That's because we didn't do a
good enough job of managing the complexity as we added new
things. So focusing all of the time on
managing the complexity flattens that cost of change curve
retains at Freedom, to change our minds and to do new things
and new features. Modify the features that are
already there more easily. It might be difficult work
sometimes but we still have the ability to do that then if we
build some big complicated coupled ball of mud that we're
afraid to touch so managing the complexities about trying to
organize our thinking universities that
compartmentalize the problem it's a bit like writing a book.
If I wrote a book that was just one long sentence it might have
all of the same words in it but it'd be much much harder to read
than if I break. Get down into sentences and
paragraphs and chapters and sections and label that gives us
markers of where we are. We can jump to a play a
different part in the book. And look at only that part in
the book, I'm going to refer back to those are book level
tools for managing complexity ideas like modularity.
Cohesion, separation of concerns abstraction and coupling.
Our software levels are quite ideas for managing complexity to
be fair. I apply these not just to
software, but to Information Systems in general the way Open
eyes, my hard disk, he's kind of simpler returns of modularity,
and cohesion and trying to break things into pieces, and keep
things that are related close together and things that are
unrelated to our part of my dish, because it makes it easier
for me to navigate the system. So, I think these ideas are
foundational principles that we should apply to basically
everywhere I'd be talking to some people building packaged
software and low coat software this.
Treatable still apply. There are be talking to people
to do machine learning these principles, still apply it.
I think those are the kinds of things that we would expect if
we were to identify genuinely important principles for an
engineering discipline. That was related to software.
Yeah. I like the way that these five
key principles can be actually taken not just from low-level
coat but actually do something bigger like machine learning and
sometimes getting this distributed systems and other
you cover about why microservice exist, what microservice exists
is partly also by adopting all these key principles, there are
two things. When we build a complex may be
distributed. Stubbs right?
You mentioned that two things that probably is root cause of
more complexities about concurrency.
And coupling sometimes these are unconscious though, when we
build software we don't think about multi cam current process
at one go. And also coupling, it could be
dependency coupling, coupling between classes or even
competing with other services. Maybe if you can touch on these
two primary cause of complexity in the modern software.
Yeah, we pleasure. I think those two things, it
independently but certainly can bind our kind of are equivalent
to come Quantum physics. They're incredibly complicated
and we are always just moments away from which is interesting.
As soon as you create a branch in your version control system
or spawn a new thread, you've just jumped into that world of
complexity of concurrency and coupling between those
concurrent systems. And it's always comes at a much
bigger cause they're not doing and it's go back to it analogy
of writing a book if you and I are collaborating.
Writing a book then we could divide up the work and you write
one chapter and I write another or if we working on the same
piece of text you could do so that we could send this changes
between each other and all that kind of stuff.
Or we could put it on google doc and we could work on it could
currently we could see our changes that last one is a much
better way of working and much more efficient way of will,
because we can see what's going on and we get that fast
feedback, we can learn really quickly.
Oh, you're working on that part Henry's, working on that part.
I'll avoid that for now while he's working on, whatever it is.
So we get much better feedback those way.
So thinking about those kinds of ideas and how the information
works is important in the latter part of my career, when I was
Building Systems, right? Talking buildings, I was
involved in building what we think was probably the world's
highest performance Financial exchange and that was
fascinating. We had to do some not
necessarily me but the team as a whole had to do some really very
clever be things in terms of deep concurrency and man.
Shooting the flow of information to have to get the levels of
performance of your base to achieve.
We were looking at the specific architecture of Intel processors
and designing software that would take advantage of that to
maximize the throughput of our Co that's very difficult stuff
and it's the same difficult stuff that we have.
As soon as we spawn a new thread, that needs to join up
with another trade, the cost, we did some benchmarking kind of
things to look at this. If you just increment the number
and you say, oh, I mean creating this.
It's a very large value so I'll speed it up by making it to do
in parallel if you just do that. Then the cost of the most
efficient work of synchronizing the results back so that you get
a valid result at the end is 300 times slower than just doing on
a single thread. So there are naive ways of
optimizing things and there are clever ways of optimizing fixed
parallelism. He's very valuable for certain
kinds of problems and very stupid or other kinds of
problems one. The places where this surface is
high enough, recognizable places information control.
If you have a relatively coupled system and you divide that
system up into a bunch of microservices, each living in a
separate repository but you don't trust them enough to melt
and release them independently. So you've got to test them all
together before you're safe to release.
That's one of the more inefficient ways of organizing.
Their if you put them all into one big repository, there are
different kinds of optimizations that you can do then if they're
inseparable. Because the trees you are adding
a bunch of overhead to your ability to go in and change
coding different repos, all of the time.
So it slows you down. There are lots of advantages to
microservices that they will give you but this is a problem
of concurrency and coupling between the these I think the
concurrency and coupling is the hardest part of software
computer science. Really everything else is
relatively straightforward in comparison to those two things.
So being defensive about concurrency and coupling is a
very Strategy, a couple of my friends are considered to be
world-class experts include current system.
Their advice is always dumped a currency unless you must, they
are the world's experts they say, don't do it unless you
must. So I think that being defensive
about those sorts of things and managing those things.
And one of the things that we do that alma mater, we built our
very high performance exchange was that we are conceptually
isolated concurrency, so that didn't happen.
Where we were doing the business logic.
So the business Logic for our system is very, very high
throughput, very widely used hundreds of thousands of users.
All the business logic was process on a single thread for
each piece of business lunch, but we did smark things in being
able to bring the information to the point at which those single
threads could process, the logic.
So we externalize separated the concerns.
I've been able to do that. So, I think there are smart ways
of doing this in organizing, these kind of things, but I
think concurrence in coupling are the enemy.
You should be doing everything else to kind of God.
A Celtics, those things that's hard to be engineering
discipline part in managing complexity, thanks for sharing
this insights. I'm always amazed when in your
book you mention about lmax. This is like a very high state
of the art kind of system. Some of the principles also that
you mention is it's not really just optimizing for Speed
paralyzing and all that your could must be readable.
But when you isolate the concurrency such that, you can
still make changes and still the system is performing.
It's not like optimizing syntax or lines of code or try to make
assembly code better. Absolutely, 100%.
One of the things that we looked at measure L Max was the
readable code, is nearly always more efficient, not because it's
not that, it's an overhead into because if the code is readable
you're going to be striving to make it simple.
So it's expressive and clear what's going on and compilers,
love that shit. If the code is readable to us
then it's readable to compiler. Therefore the compiler could go
to town and optimize it more effective but more than that if
it's read. Abel.
It's going to be simpler and so there are fewer places there for
nasty problems, too high. If you think about what high
performance software really is, it's about delivering the most
impact for the fewest instruction.
So that's kind of definition of what we would think of.
And so designing, for Simplicity is really what we want.
Simplicity is really what gives us the readability.
Readability is a fantastic tool that will drive us in the
correct direction. You've got to do other things so
I could formats as well. But readability, I think is one
of the most profoundly important tools.
For all of those disputes of managing complexity.
He makes that go more modular more cohesive, we'd better
separation of concerns more lasting those things.
Make our code more readable. We can build a little sentences
up as we create our cone, but even a non-technical program, I
would be able to understand what was going on if you get the
names, right? So I think you're right that
readability is profoundly important.
One of the real import Tools in our toolbox.
And also on decoupling, right? The other source of complexity.
So you mentioned just now if you write microservice but you
cannot independently release it or independently test it, this
is also some pitfalls that people do they write
microservice but before release they need to coordinate multiple
Services working together and do the regression.
So I learned from somewhere as well, that coupling is one of
the killer of age Unity. So if you cannot manage
complexity, basically, you cannot have this agility.
Thanks for emphasizing this concept.
So maybe if we Sometimes to cover the tools, the tools that
you mentioned can be the modern software engineering tools.
What are those actually tell you mentioned about tdd continuous
delivery? Maybe, there are some other
things that we haven't learned from this conversation so far.
Yeah. So there are a few other ideas.
One of them is again, going back to related to the door report if
we optimize our development approach to be able to work in
smaller steps. So get faster feedback on our
changes, we get to add more definitive position we can make
A small change. Look at the effect of that
change and understand it. Technically, I think one of the
reasons why that is important is because that's one way of
limiting the variables, if we can make a small change and
evaluate that change essentially in its own context without
worrying about the whole, the rest of the system that's going
to give us a clearer picture of when the Chinese good old not as
long as that system is modular and Loosely coupled and all of
those other things. So That's a good way of working
in continuously returns speed is probably one of the most
profound tools that I use in trying to help big organizations
to transition to working with continuous delivery.
You just start saying. Okay, so where are you now?
You're releasing once a year. So let's try Andre.
What would it take for us to releasing once every six months?
Let's do that. Then every three months, then
every month, then every two weeks, then every week, and just
keep finding what the problems are that get in the way you use
the speed of Feedback release ability to drive down and remove
work and complexity from the development process until you
can release every day. That's in great way of doing
things. It's a great way of thinking
about and structuring and exercising changing the way that
people in organizations worth deployability is another of
these fantastic tools and talking about micro service
means kind of points to that. Very clear if we want to be able
to take an engineering stats, they're really what we'd like to
be able to do is to be as Trinity of as we possibly can
about our evaluation of our software before release.
So I'd like to evaluate my software and have as much
confidence as I can achieve before.
I release it not just put any old rubbish into production and
red fruits fall over. I'd like to be able to evaluate
it in as close to the circumstance and says I can
imagine for it to be Deployable. There are two strategies as far
as I can. See that make sense to be able
to do that. Both of them are driven by the
deployability for supper. Anyway, which is why I could
deploy guns is a useful tool. So one of them is that we could
just decide that we're going to treat our whole system, as one
thing will evaluate it all together.
If all of our tests pass, you'll say, yes, it's good.
And we'll put into production, or we could decompose it into
many indications of Parts. Evaluate these parts and say,
this part is good and now that go to production.
The second one only works as long as those things are
independent. Deployable if at the end of my
deployment pipeline, Say this part is good but now I'm going
to test it with all of these other parts before I release it
then the real pipeline. He's that second one where I'll
be evaluating everything together to be able to do
continuously. You have to be able to do that
once today and there's a lot of complexity now because I've got
all these separate, Repose and separate deployment pipelines.
And lots of in efficiencies, with that way of working,
microservices are independently Deployable.
So I think you can use the deployability in the system as a
tool. So we can say, we want to
release change into production. What side?
Humans have deployment. What is this thing that I can
evaluate to the point where I'm happy for it to go into
production without doing any more work?
What's that unit? That tells me the scope of your
valuation of my deployment Pipeline and probably the scope
W, goes into my repository. That's a useful at all the
different phenotypic statement on how you structure your code
in repositories and things like that scoping evaluation needs.
They deploy or unit. All of these things, test-driven
development Speed release. Ability.
All of these things are tools that we can use to amplify our
ability to learn and our abilities as complexity of the
systems that you build. So they kind of come together
and that way of working. Amplifies, those software
engineering properties, that we're striving to achieve.
I like the way you mention about the scope of the pipeline.
So in your book, you mentioned that.
If your pipeline says it's good, then it's good to go.
There's nothing else that you should wait, no approval.
No coordination with other teams.
I think that's also another key. Insights from me, when I read
the book is because sometimes when we have a pipeline, okay?
All test, run successful and all that, but we still wait the
approval of waiting other system to release verse or something
like that. But the concept of good pipeline
is that when it's good, you just release it, there's nothing else
that you should react. So that's one of those semantics
diffusions that we were talking about earlier.
I am the person that came up with the idea of the deployment
pipeline. So that's absolutely what I bet.
He sees Deployable at the end of the pipeline, I tend to work in
more complicated Industries. These Lisa regulated Industries
building medical devices or finite systems or Machinery.
The challenge there is, oh yes. But we got to get this sign of
buying supplies. Know you can automate that as
well as a slur made a change to the model 3 production line.
This is a chase the factory. They upgrade the charging for
the modern three come from 200 kilowatts to 250 kilowatts that
was a software change which reconfigured the factory to
build cars differently and it took Took three hours.
Wow, so they put this upgrade change in, he went through their
deployment Pipeline and then the fatuous produces that you first
about Pre-K, SpaceX are changing software on their Rockets, 43
minutes before the rocket launches.
Wow. So you can do this, you can do
Mission even very difficult circumstances, but it does take
a different mindset. What I'm thinking was an
engineering mindset to do. So, I'll suffer developers who
listen here. If you think doings Liberty is
hard for your software, think about SpaceX and Tesla and how
they can do it with class Factory and Rockets.
So I think we can all aspire to become much better, and try to
build better software faster. So they've thank you so much for
sharing your insights all these conversations.
I think it's really great. But unfortunately, due to time,
we have to wrap up pretty soon. But I would like to ask you one
last question before I let you go.
So, this is something what I call tree.
Technical leadership, wisdom, something like advice from you
to all of us here. This Learners to learn from your
journey or experience or from your past projects.
So what will be, Dave's, three technical leadership, is them?
It's a good question. I think.
My first one, I think the industry as a whole would be
significantly better. If we all started out, assuming
the are own ideas were wrong, rather than assuming that they
were, right? So going thinking, this is been
to be rung, but I wonder how it's wrong.
A kick. That's an engineering mindset.
That sets us up for Control the ways in which our engineering
mindset is wrong, thinking about the ways in which our software
on our system isn't working practices can go wrong.
I think that's a much better way of organizing thing.
If I could flip a switch and change the way that software
developed worked, I would make it so that everybody from now on
that learned how to program, did it in the context of test-driven
development first, their first day of writing their first
program, they started with a test.
That's the Way that I would teach test programming, because
I think that if we learned people like that, there'd be
none of these arguments about whether it was Bachelor, not
because we know that it is. So I think that's the next one.
The last one is I feel privileged to found a career
that I have reveled. I have stained consistently even
on the difficult days, even when I'm struggling, I loved what
I've done for a living. It's fantastic.
I think that we should sometimes stop and think about that.
And think about The value that we bring to the world, we are
the engines of change in our society more than almost any
other group of people. That means that we have the
privilege of being in that position to do amazing things,
like pools, and social media, Rigby internet, and all of that
kind of stuff, which is fantastic on many fronts, but
it's also means we have a responsibility to do that in a
thoughtful way and worry about some of the implications of some
of these things. So I would also Also ask people
to be a little thoughtful about the context in which they work
his place, and that's not somebody else's responsibility.
The Volvo emissions Scandal. A few years ago, showed that
saying that my boss told me to do, this is not an excuse, the
person who did it, still went to jail.
So we should be taking responsibility for our own
working, speak about to see the social context to understand the
check. Well, that's really beautiful.
I was guessing that you probably will say something about
continously. Rebut that Lee is something else
though Dave, I'm sure that people know where to look for
you, but for completeness sake, if people want to connect with
you and continue this conversation where they can find
you online, you can get me at Twitter, my Twitter handle is
apps. They slowly 77, go to YouTube
and search for continuous delivery and you will find my
Chapel. There's some good stuff there,
even though I say so much, that my books are available through
the use of sources, you can buy all of my books on Amazon, one
of them at least on leap. So, take a look for those, as
well. Have a blog site as well date
following. So in preparation of this
composition, I must say that your book is really insightful,
and thank you for writing that. I find it really good.
If we all Engineers wants to upskill and maybe go back to the
basics a little bit so thinking about learning and also managing
complexity. So, thank you so much today for
your time today. It was a pleasure.
Thank you so much fun. It's not choking Thank you for
listening to this episode and for staying, right until the end
if you highly enjoyed it. I would appreciate if you share
it with your friends and colleagues who you think would
also benefit from listening to this episode.
And if you're new to the podcast, make sure to subscribe
and leave me your valuable review and feedback.
It helps me a lot. In order to grow this podcast
better. You can also find the full show
notes of this conversation on the episode page, at Tech Legion
o.f website, including the full transcript enter, In quotes and
links to the resources mentioned, from the
conversation. And lastly, make sure to
subscribe to the shows mailing list on pack leader dot f to get
notified for any future episodes.
Stay tuned for the next technology.
No episode. And until then goodbye.