Those are our four key metrics deployment, frequency the time
for changes change, failure rate and your time to restore
service. Now, what we found as we looked
at those four metrics and what we found consistently over
throughout the course of the research study is that they are
trade-offs many organizations think.
In order to be safe, I have to be slow but the data shows us
that the best performers are getting both and in fact as
speed increases so too does. T.
Hey everyone. My name is Henry Surya Barragan.
And you're listening to the tekhelet Juno, the show will be
bringing you the greatest technical leaders practitioners
and thought leaders in the industry to discuss about their
Journey ideas and practices that we all can learn and apply to
build a highly performing technical team and to make an
impact in your personal work. So let's dive into our Journal.
Hello. Hello to all of you, my friends.
I hope that you're feeling great and healthy today.
Welcome to another new episode of the technology on our
podcast. Thank you for tuning in and
listening to this episode. If you are new to technology,
you know, I invite you to subscribe and follow the show on
your podcast app or I will growing social media communities
on LinkedIn, Twitter, and Instagram.
And if you have been regularly listening, and enjoying this
podcast, and love the type of content that I'm producing, Will
you support the show by subscribing as a patron at
technology? Not Dev slash Patron and support
my journey to produce great episode.
Every week state of devops report.
Many of you may have heard about this report which has been
running for over the past seven years.
It is the largest and longest-running research that
aims to provide data driven industry, insights, that examine
the capabilities and practices that drive software delivery as
well as operational and organizational.
Final performance. My guest for today's episode is
Nathan Harvey. Nathan is a developer Advocate
at Google, a part of the Google Cloud Dora research team and the
co-author of the 2021 accelerate state of devops report.
He was also the editor for the 97 things.
Every cloud engineer should know, published by O'Reilly in
2020. In this episode, we discussed in
depth, the latest release of the state of devops report, Nathan
started by describing what the report is all about,
historically, how it got started and what problems it was trying
to solve and then he explained the famous five key metrics
suggested by the report to measure the software delivery
and operational performance, Nathan then explained how the
report categorises different Performance Based on their
performance against the key metrics?
And how the Elite Performance, outperform the others in terms
of speed, stability and reliability.
Next, we dived into several new key findings.
That came out of the 2021 report that relate to the importance of
good documentation. Secure software, supply chain
and positive team culture. That mitigates Bernard during
challenging circumstances, like the pandemic towards the end,
Nathan gave great tips on how we can use the findings from the
reports, not just the latest, Report on how we can all get
started and improve our software delivery, and operational
performance. That ultimately will also
improve our organizational performance as a whole.
I really enjoyed my conversation with Nathan learning about the
accelerate state of devops report, the five key metrics,
and the new key findings that we can find in the latest 2021
report. And if you also enjoyed this
episode, I encourage you to share it to someone, you know,
who would also benefit from it. You can also leave a rating and
View on your podcast app or share some comments on the
social media on what you enjoy from this episode.
I always love reading your reviews and comments and they
are one of the best ways to help me spread this podcast to more
people. And it is my hope that it can
also benefit from this podcast. Let's get our episode started
right after our sponsor message. Are you looking for a new cool
swag? Taglit Journal.
Now offers you some swags that you can purchase online?
These wax are printed on demand based on your Reference and will
be delivered safely to you all over the world where shipping is
available. Check out all the cool tracks
available by visiting technology, you know, dot, f /
shop, and don't forget to break yourself.
Once you receive any of those tracks, hello everyone.
Welcome back to another episode of the technology, you know,
podcast. Today I have with me, a
developer relations at Google. His name is Nathan Harvey.
He's actually one of the co-author of the latest state of
But 2021. So if you are not aware about
state of their jobs report today we'll be covering a lot about
it. Talk about what are the latest
findings as well. Nathan is also, someone who is
experienced in develops, a sorry.
So, he'll be talking a lot. Probably from his experience as
well. And he's also an editor for the
97 things. Every car engineer should know
which is published in 2020. So, welcome to the show.
Nathan looking forward to have this discussion with you about
state of devops report. Oh, thank you so much, Henry,
and I'm really happy to be here. So Nathan in the beginning.
Maybe if you can introduce yourself sharing more about your
highlights or turning points in your career.
Yeah, for sure. Thanks.
As Henry mentioned. My name is Nick and Harvey our
developer relations engineer here with Google Cloud.
Some highlights of my career. Well, it's been a long career,
for sure. I would say, the couple of quick
highlights. I really look at my career, as a
career, where I've tried to help teams improve the work that
they're doing, sometimes that means helping teams improve the
work that we are doing for, Example, I've worked with many
different web applications. Well actually, one of the first
applications I was responsible for Henry, we called it the
customer and partner Extranet. Do you know what an Extranet is?
It's basically a website, but you have to login to get to it
in the late 90s, early 2000, it was mine.
Breaking that we would have these things on the web for
customer. So, I've been a lead engineer on
a project like that I've helped an organization.
A team that I was on really at adopt automation practices
moving things from data centers into the cloud.
One. Turning point that I have is I'm
working at this huge software company at the time.
I was responsible for managing our sales force automation tool
which happen to be a product called siebel back in the day
and it ran in our data centers and I recognized the problem we
needed more capacity for our servers but they run in our data
center. So I went to my boss and I said,
Boston need more capacity and he said yeah.
Fill out this form and then I'll get it to accounting and we'll
get it all approved. So filled out the forums, got it
all approved Henry, it was only 18 months later that I was able
to log on to that server was a very, very fast 18 months.
That's all it took. Obviously that's unacceptable in
today's technology landscape the very next job.
I went to though was a start-up and we were doing everything on
the cloud. So instead of waiting 18 months,
I could put my credit card in to a web form and in five minutes
it's have access to a full server.
A server that I would never be able to touch but I have full
control over that server. That really changed everything
about how I approach Taco. I think about tech it was pretty
incredible and then at that same organization I was responsible
for operations. Like I said, it was a start-up,
it was very small. I was responsible for everything
and have a good Mentor who was helping me one day that Mentor
came to me and he said, Nathan, you know, there's these tools
out there that can automate a lot of the work that Doing maybe
you should look into them. They have names like, puppet and
Chef. And I looked at him and I said,
you know what, Tom, I am too busy to automate too busy to
automate, what a great phrase, and that phrase is really stuck
with him, too busy to improve. Obviously, if you automate,
you'll have more time and more capacity.
So, I took those words to heart and eventually I learned both
puppet and Chef and ended up using Chef quite a lot in
another organization. And then eventually I fell in
love. With the chef Community.
I have to tell you it was open source and it was a community of
practitioners that were really automating infrastructure.
I felt so much in love with that Community.
I said, I have to help that community.
And so, I joined Chef the company and I was part of
leading and fostering that Community for about six years.
After my time at Chef, I joined Google Cloud to continue my
journey through devops into a sorry and really helping teams
find ways that they can perform the best.
Thanks so much for sharing your story.
I could still remember as well traditionally last time we used
to ask for servers and all that even hardening process itself
and take months. So I think I really relate to
your story, and thanks for sharing the story where you
said, that you are too busy to automate.
I think we have been there, especially all these as arjan,
Dev Ops people. Sometimes we are just too busy
in our daily work. They'll eat oils and also forget
how to actually improve from there.
It's Sookie. It's Sookie.
Even to make a small investment in improving, just a little bit
better. Every day.
Speaking about Improvement and devops related automation this
year. I saw that Google cloud and Dora
just released a state of devops report 2021.
So can you tell us more what is actually state of devops report?
Yes, absolutely. So the state of devops report is
essentially what it says on the title.
It's a state of devops report about this but let's get into a
little bit deeper background there.
So there's a research program. Dora research program, it is the
longest and largest running research program of its kind.
Dora itself, brings an academic rigor to the investigation of
this question of how two teams improve their software, delivery
and operations performance. And so each year Dora has
published a state of devops report that summarizes the
findings from that years research.
From that years investigation, the investigations, the
research, always go across Across every industry vertical.
So we talk to organizations in finance in government and
technology in retail across the board really seeking to
understand what helps teams improve.
So, if I'm not mistaken, this has been running for the last
seven eight years is that correct?
Yeah. Yes, that's right.
It's been seven years now that this has been running.
Yes. So if you can, probably explain
how did they started? What kind of problem that they
are trying to solve back then? And why?
Coming up at this report itself can help ya.
Interestingly the report itself in their research actually was
started by Puppet Labs or a puppet and what they saw.
Almost a decade or over a decade ago.
Now, they saw this burgeoning community that identified as the
devops community, puppet had this question, what is the
devops community? What are they doing?
How can we bring the lessons that Community is learned to a
wider audience? So they started the state of
devops report and an annual survey to go with that.
Now, as that program evolved dr. Nicole for his grin, got
involved in that to again, bring a little bit more academic rigor
to the research and to the investigations.
So those reports continue to then eventually dr.
Nicole for is Rin together with jazz humble and Gene Kim formed,
an organization called Dora, Dora stands for devops research
and assessment what that organization did was continue.
The research program in conjunction with Papa And on
their own. But they also built a tool, an
assessment tool to go in and help organizations measure, how
are they doing against their software, delivery and
operations performance? And importantly, how to
identify, where they should focus their improvement.
Work Dora eventually, in 2018 was acquired by Google cloud.
And we've continued to expand on that research and continue to
work together to bring the findings of that research and
practical implementations. Is to our customers.
If I can also relate with my personal experience, looking at
these reporter. I think since the state of Davos
reports suddenly we come to see a lot of state of other things
reports in the market, including actually lately that we have
seen puppet. Also releasing another state of
devops report. Why are there so many reports
maybe can you clarify little bit here?
Yeah. So why are there so many
reports? Well I think we have a lot to
learn from one another and there are always questions that we
have. So I think this is really
important. I think when Dora the
organization was formed and started publishing their own
reports Papa continued with their own research.
I was just looking at the 2021 puppet state of devops report
and I found fascinating insights in there.
The methodology is similar across the two, although not
identical. And certainly the things that
are investigated in each Year's report are completely different
to one another. So I think it's really
interesting, but I also think, you know, we see the state of of
the secure supply chain or the software supply chain, we see
that a state of that report, the state of Dempsey cops.
Yes, there were reports everywhere.
And I think the important thing to remember, is how do you as a
team utilize, the findings of a report to help your team
improve? That's the biggest question,
always thanks for putting that as a note because we all see
these report we all see the findings but at the end, I don't
know whether we improve or not. So that's the only measurable
things that we need to do speaking about the fine.
We know that state of Davos report has this popular
findings, which was used to be called for Golden Matrix of four
key metrics. But now there are five key
metrics and you probably explain to us.
What are these? Five key metrics?
Yes. Absolutely.
So the five key metrics and you're right they started off as
for key metrics really at the beginning of the research
program. The question was, how do we
improve and sure software delivery performance, frankly
when it comes to the word? Devops Henry, you have a
different definition than I have, then any one of our
listeners has this is one of the challenges of devops as a
community. Devops has always shunned a
common definition. So as a researcher, how do you
measure a thing and research a thing that's not?
Well defined. Well this was one of the
problems that the door research team set out to solve for.
So what they came up with was a very practical and pragmatic way
to measure. Our software delivery
performance, and that's where we got the Four Keys.
So, software delivery, how do you measure software delivery?
Well with Endora, what we measure, our two, metrics for
stability, and two metrics for throughput.
I'll start with the throughput, the velocity.
How fast do you delivering software for that?
We look at deployment frequency. So how frequently are you
pushing changes out to your customers or the users of your
application? And then importantly, we look at
lead time for changes So as a developer, I'm making a change
to the code base, I commit that change to the Version Control
System, you're using a Version Control System, right?
Henry, I hope all of you are, if you're not using a Version
Control System, stop listening to this podcast go read up on
get or any other version control system and then come back and
listen to the rest of this podcast, okay?
Sorry I had to get on there for a second so to play with
frequency how frequently re pushing out changes.
Next is lead time for changes as a developer, I make a change,
commit it to the Version Control.
Timer starts that timer stops when that change lands in
production and so we want to increase the frequency with
which were pushing out changes and decrease.
The amount of time it takes for those changes to go from the
developer into production. Those are our velocity metrics
and then we have our stability metrics, moving fast as great.
But if you're moving fast and always breaking everything,
that's not so good. So stability.
We look at two things there. The first is your change fail
rate. So when you make a push into
production, how frequently are you shouting out an expletive?
Oh my gosh. Ah this is broken.
We have to roll it back immediately.
So when you have that, that's a change failure.
Someone had to get involved in that deployment process.
We want to decrease those as much as possible.
Our final stability metric has to do with this idea.
This truth that it doesn't matter.
What we do to reduce risk risk is always present failures.
Inevitable. Something is going to break so
we have to look beyond the tools.
We have to look at our team and say how quickly does it take us
or how quickly can we restore service to our customers?
When there is something that impacts them.
And so this is our time to restore service.
So those are our four key metrics deployment frequency
lead time for changes change, failure rate and your time to
restore service. Now, what we found as we looked
at those four metrics and what we found consistently over
throughout the course of the research study is that they are
trade-offs many organizations think.
In order to be safe, I have to be slow, but the data shows us
that the best performers are getting both and in fact, as
speed increases, so too does stability.
So they have some really interesting findings there.
Okay. But you asked about the five
metrics and I said there was a fifth one that we've added this
year that fifth one that we've added, we've really refined it
this year. I would say for the past couple
of years since I want to say, 18:11 might have to check my
notes on that 2018 2019, whatever it takes.
We introduced this idea, not just of software delivery
performance but software delivery in operations
performance. Now operations performance.
Because when you've deployed something that's the beginning
of it, providing value to you as an organization and to your
customers. So we really need to answer this
question of what's our operational performance look
like in the past in previous years, we asked questions about
your operational performance. It's in terms of availability
but this year, we've refined our approach and gone deeper with
the research to move from, availability to reliability now.
A subtle change in language but as we know language is
important. But that subtle change is really
more to help everyone. Consider not just is the
application available but availability is really a
component of reliability. In both cases, though.
What we're really trying to answer.
Question. We're getting at.
Can you keep your promises to your customers?
Can they rely on the application or the service that your team is
building and shipping? So what we found this year, as
we dug, really deep into reliability.
Is that reliability practices, push organizational performance.
They drive that organizational performance as an organization.
If you want to get better investing in reliability can
really help. I really love the way you
explain all these key metrics, hey, with a little bit of humor,
he had asthma. So this is my first time here is
such a hundred of them, golden key metrics.
So just to recap, that's the philosophy of through booked a
metrics, which are the deployment frequency, and also
lead time for changes daddy stability, part, which is the
change, failure rate and tied to restore the service.
And the fifth one, which is related to operational site,
which is now called reliability aspect.
So it used to be Ability. But now we change the turn to
reliability. I think you mentioned as well in
your explanation that used to be traditionally well-known, right?
So if you want to be safe for reservist, you have to go slow.
So that's why there are maybe quarter release monthly release
or even year we released because the software just couldn't get
deployed. And now, it seems that most of
the organization's understand that in order to be more safe,
you have to go faster. And that's why you have this
velocity and throughput measurements to actually
indicates whether the organization.
Changes rapidly or not and now you have the reliability aspect
as well. What does it tell the balance
trade-off between risk. And what's the operational side
actually tells us? Yeah.
Interestingly, what we found is that all of the performers
across, whether you're low performer, were the best of the
best, the elite performers when it comes to software delivery,
each one of those performance categories, could improve their
overall organizational performance with the adoption of
reliability practices. So we find that these are Very
complimentary practices to one another.
And in fact what we talked about or what?
We investigated a lot of within reliability.
A classic example here, are you using the data from your
reliability measures to help you?
Prioritize work in the world of site, reliability, engineering,
or the practice of a sorry, we often talk about an error budget
and we use an error budget to say if we're not meeting our
reliability goals and we should focus our Effort on the things
that will make the system more reliable and flip side of that.
If we are meeting or even exceeding, our reliability
goals, then we should focus on things that make us faster.
The other thing I think that's really interesting about all of
these metrics together. There's kind of two things I
want to point out one. We have to make sure that as a
team responsible for an application or service, we have
joint accountability to all of these metrics part of the reason
in the past, we used to think, That slow met safe.
It also comes down to who's incentivised for which we've
spent a lot of times telling our developers go faster, go faster,
and then turning to our operators and say, keep it
stable and I've been that operator.
I know the best way to keep the system stable is to not change
it but that's no good for our customers, right?
So we have to take these five metrics are for software
delivery metrics and our operational metric and really
have joint ownership of them across with all of our teams.
The other thing that I think is really interesting is just how
do they reinforce one another and it devops technology
practices. We learn from other disciplines
as well. And I think the biggest lesson
or one of the biggest lessons that we've taken from another
industry, is that from lean manufacturing and lean
manufacturing says, do things in small batches.
So when we apply that to the delivery and operations of
software, we know that when we have a small change it, Always
heading to production. That's a small change.
It's easier to reason about when it lands in production, it's
less likely to have a huge negative impact.
We also bring in practices like Progressive, deploys, where you
might deploy that change 25 percent of your users, and see,
how did that go before you roll it out, globally, to all of your
users by making those smaller changes, you're less likely to
have a big impact, your more easy to reason about it and move
that. Change through the system as for
those two summary because that actually explains the gist of
the devops itself. So when we traditionally used to
have fights between developers and operate this now with the
devops concept and culture. So basically those two teams
need to have a joint account ability, like what you said and
also incentivised to actually work together, make the
customers happy at the end of the day, speaking about how the
state of that was report classify, what they call
performers, right? So they are few categories.
They are the elites and the low. Up to low performers.
So can you explain a little bit? What are these categories?
And what do you see, the significant difference between
Elite and maybe low performance? Yes.
For sure. So again this kind of gets back
to the sort of academic rigor, of the research itself.
So what we've done each year, as part of the research program is
asked respondents about how were they doing against those four
key metrics, or those five key metrics.
Now, of course, and what we do is we take their answers and we
throw them down onto a plot essentially.
So we lay them all out on a grid.
And we see these clusters emerge, these clusters, show us
that there were three statistically significant, or
for statistically, significant groups of performers.
I said three first because actually, in the beginning, part
of the research program itself, there were three different
distinct groups but in 2018, a fourth group started to emerge,
but subset of those top performers.
What we've seen are or what we decided to do in 2018 was
identifying call those out as Elite performers.
So today and since 2018, we've had low, medium high, and delete
performers, four different clusters of performance.
So what we do is we look at how are they doing across each of
those metrics? So at a very high level, I'll
tell you a little bit about the elite performers in the low
performers on that deployment frequency.
Elite performers are able to deploy on Demand, that may mean
even multiple deploys per day, but a low performer in 2021 will
do fewer. Deployments, then once per six
months, so it might take seven eight months maybe yearly for
them to do a single code deployment.
The flip side of that of course is our time to restore service
or change fail rate on the time to restore service.
I'll give you the Deltas there. Between the two time to restore
service for an elite performer less than an hour for a low
performer. Take more than six months to
fully restore service. When there is an outage or
something that interrupts their. Now, the other thing about those
four categories is because we've had them for many years, we can
see changes in the shape of those.
So what defined Elite performers?
Three years ago, maybe only defines High performers today
because as an industry we are always getting better.
We've also seen that in 2018, the elite performers were about
seven percent of the sample. In 2021, they're 26 percent of
the sample. So that lead performers, not
only, are they getting better, but the number of people are
number of teams that are able to achieve that is growing.
Wow, that's a very Stark difference between the elite and
low performers. So if any of you are interested
to assess yourself or your team where you are at, go check the
state of devops report. I think each of the four Matrix,
you will have some statistics, what will be each of the
performer? Kind of like performing at that
particular Tricks. So, for example, just now Nathan
mention about deployment frequency where Elite
Performance can do only one multiple deploys per day.
So the low performers probably somewhere around six months for
deployment frequencies and that's really bad if you can
classify yourself. Of course measure where you are
at now and you can look forward how to improve it.
That's kind of like the roadmap out of these findings.
Absolutely. If you can absolutely check the
state of devops report but we also have a tool Henry that's
available to anyone. If you go to cloud.google.com.
Cam devops. You'll see there's a button
there that allows you to take a quick check.
So this quick Jack is the Dora, devops Quick Check and we will
ask you questions about your team's performance today and
help you assess. Where are you?
You can then compare yourself to other Elite performers or high
performers. You can compare yourself to
others within your industry and across all of the survey
respondents. So definitely check out the
quick check, thanks for the plug they.
I'll make sure to put The show notes.
I wasn't aware of that tool, but yeah, definitely, it will be
great if there's an automated tool that can help you to
assess. Yes, so speaking about this
year's report, what are the new key findings?
I saw a couple of new key findings.
Maybe you can share. What are the most important
significant findings this year? Yeah, absolutely.
Because when we look at these clusters of performance, that's
really good to know and help understand where you are.
But the real question is, how do I get better?
That's really where I find the research to be very powerful.
Powerful, because we dig into different capabilities.
The teams should invest in. Now, those capabilities might be
technical capabilities, but they're also process
capabilities, measurement, capabilities, and cultural
capabilities. We really want to take a full
systems view, not just that systems, like the computers, but
the systems that include you and me that people in the system
Henry where they're. So at a very high level, there
are a number of different things.
That we investigated this year, we touched briefly on a sorry,
so SRE and devops. We wanted to investigate.
What's the difference between them?
Are they working together? And in fact, we found that they
are complementary. We also continued our Research
into Cloud, not surprisingly. We find that cloud is one of
those capabilities that drives performance but it's not just
paying a cloud bill. You have to do Cloud the right
way. And in fact, we lean on other
research from the national Toot of standards and technology,
which helps to find the key characteristics of cloud
computing. The other things that we
investigated security, no surprise here.
Security is on everyone's mind right now, especially if you
look back over the last 12 months, 24 months, the number of
attacks and vulnerabilities, that are out there were all
thinking about security a lot. And so, we dug into that.
And then another one that was really interesting this year,
that's brand-new this year, in fact.
Is we looked at documentation? Everyone knows documentation.
We need to do better with documentation.
Well, we wanted to make sure that was actually true.
Yeah, we feel like Doc's could be better or that these docks
are bad, but does documentation actually drive performance.
And so we're able to draw a predictive correlation or a
predictive connection between good, documentation, it enables
the right technical practices to help you hit those goals and
then of course, I can't talk to anyone.
About devops without talking about culture.
So, yes, of course, we investigated culture.
Specifically, we looked this year at how teams are doing with
burnout, what sort of things can we put in place to help?
Mitigate burnout, from a cultural perspective.
So, those are some of the high-level quick findings there.
And I'm happy to dig into any of them with you Henry, that by the
late 1950s, the most surprising thing is the documentation,
right? It was never before in the state
of devops report, but this year, there was this new thing.
So why do you think Documentation, can actually be a
predictive measures of how performing an organization could
be. Is it because people just need
to find a summary of the things within the company or is about
Playbook and run book? Or is there other types of
documentation? Yeah, this is a really good
question and of course or a lots and lots of different types of
documentation, we kind of investigated.
What are the practices in particular around documentation?
That matter can what we found was predicted Of connection
between things like having clear ownership and clear.
Guidelines. For when documentation should be
updated making sure that it is part of the developers daily
work. If we go back to security, we
talked about this concept of shift left shift left on
security. Make the developers or help
enable the developers to bring better security practices.
We want to do the same thing with documentation and in fact
we want our developers as they're making changes.
Are the docks being updated. This is a question that we
should be asking as part of our pipeline.
But another thing that's really important Henry is we have to
incentivize it properly, right? If I'm going to an annual
performance review and my manager discounts any work, I've
done around documentation, that's not good.
Why would I be motivated to keep the docks up-to-date?
So it goes all the way up to like incentive structures and
then down to daily practices of the team.
Is responsible for building an operating that software.
We also looked at things like when there is an outage or an
incident, do you trust the documentation?
Can you use that as a tool to help you recover and restore
service to your users? So, yeah.
Speaking about incentivizing where I think at least from my
experience, I can see that documentation sometimes and the
collected so people like to do action-oriented stuff where
documentation is used to be like well academically slow and
there's nothing really out of that.
But in this pandemic, Era where everyone probably has remote.
I think documentation is key because we want to be able to
find things properly without having to interact.
All the time meetings, these days are a lot and you'll be
getting burnt out if you always meet people and also, yes, like
what you said during optic incidents happening, do you
actually trust your from documentation?
Where? Someone else who probably not
familiar with the incident, may be able to perform the same kind
of restrictive actions without actually needing the same person
doing all over again. So I think That's really key for
documentation. How about software related
things? Is there anything about
architecture diagram or software related documentation?
Yeah, we didn't really dig in so specifically into which types of
documentation. But more on the availability of
that documentation in the process around, keeping it up to
date. But as we talk to customers as I
work with people and teams of my own experience and I'm sure
you've experienced this as well, Henry, there are lots of
different types of documentation like you.
And architecture. Diagrams are they up to date or
the architecture diagrams? Maybe they're automatically
generated from the system's themselves, maybe I don't have
to look to a document. There's documentation in the
wiki is that kept up to date. There's even documentation in
the commit messages in your version control system.
So while we didn't really investigate the various types of
documentation and try to draw any predictive connections
there, I think it is important that we as practitioners think.
About the scope of documentation.
And it really comes down to this question, right?
Of, how do we enable the best technical practices that drive,
those outcomes that we talked about this for key metrics?
And you can start to see how documentation really can play a
critical role in that? So, yeah, I mean like, those are
the examples that you mentioned architecture diagram, that can
be Auto generated. I heard some people doing
architecture decision registry or decision records.
That you also comments along with your source code, you can
also put some Malaysian in the commit messages are for people
to actually see the historical changes.
So moving on from documentation, that's another finding that I
find also interesting which is about secure software supply
chain. How do you make sure security is
embedded inside your delivery pipeline?
So can you talk more about that? How does it predict Elite
Performance? Yeah so what we found was that
the teams that are embracing and actually putting into practice
the security practices that we investigate were 1.6 times more
likely to exceed their organizational goals.
We also found that Elite performers who met or exceeded
their reliability targets. I'm going to lead performer
software delivery a meeting or exceeding.
My reliability targets. I'm twice as likely to have
security integrated into my development process.
That's huge. I think we often think of
security as the thing to slow us down and let security as the
department of. No that's not what security
should be. It's not work security.
Has to be. And in fact that when we Embrace
better, security practices, when we really shift left on
security, we see that could help us speed everything up.
So we really investigated specific practices like testing
for security. Can your developers run a test?
That would validate that the code that they've written
follows the right security policies?
Are you working together with the Security Professionals
within your organization? I really look at this as my role
is a security and Engineer or security, professional has
changed from I'm responsible for everything too.
I'm responsible for enabling the developers and enabling The
Operators to build and run more. Secure systems may be as a
security professional. I'm also responsible for
providing pre-approved code whether that's libraries or open
source packages, that week certified you can use these
without having to go through another word.
You process. These are agreed upon standards
that we can use within the organization, all of that really
helps Drive the security practices and then that
organizational performance. Yeah, I can now see the relation
with the key metrics. So, reliabilities one aspect
where security I should, he's helping a lot where if you are
more reliable, basically, you are also more secure in terms of
maybe a software deployment and maybe architecture in the sense
that people won't be able to attack it so much.
Speaking about culture, the most Bonding and devops.
So this year, there's also things about burnout and maybe
due to pandemic as well, we can see the relation to that.
So what are the key highlights of this cultural aspect this
year? Yeah, for sure.
The first thing that I want to just be very specific on the
door. Research program is about
software delivery operations. We are all globally working
under a pandemic. We leave the research to how has
the pandemic affected software delivery teams.
We leave that to the pandemic researchers but We focus on, is
this other question? So first did we see a downturn
or a drop in software delivery performance overall, we did not
see, in fact, we see teams continuing to improve.
Actually, we could look to our peers, the peers that are also
studying developers. So GitHub actually released a
report in 2020. That shows an increase in
developer activity as measured by things like pushes pull
requests reviewed, pull requests, all of this.
So even GitHub is seeing these numbers.
Improving. We also looked at this question
of burnout, how are you feeling? And really for Burnout.
One of the things that we talk about is what is your ability to
disconnect from work? Because if you can't disconnect
from work, like work is occupying, your mental space.
All the time. What we found was that teams
with a generative team culture, where a team culture, that's
really focused on the mission. Those teams that also were
composed of people who felt included, and like they, Long
time on part of their team, those teams were half as likely
to experience, burnout, even during the pandemic.
So, overall, we saw burnt-out go up but we saw that those teams
with generative culture, where the individuals on the team felt
included, like they belong to this part of the team that cut
burnout in half. So I can also relate to it.
One of the episodes that I have in the past, with the present,
also covers about the research that done by Google.
So things like generative culture, you mentioned Right
things like being included, have a sense of mission,
psychological safety, and things like that.
So I think if people wants to improve their cultural aspect as
well, I'll make sure to put that in the show notes as well so
that you can also refer how to improve it.
We're talking a lot about the findings, the key Matrix.
I know that some people may not be there in terms of performance
and all that. So what will be some of your key
advice for them to get started? Or where should they improve
first? Yeah, this is a really good
question, unfortunately. Henry, the answer is Where
should they improve first? It depends, but the door
framework can help you with this.
Because what we actually do is investigate these various
capabilities and help draw those predictive connections.
So I can say, for example, that we know that, if we want to
improve our organizational performance software, delivery
and operations, performance drives, organizational
performance. Well, how do we get better at
software delivery and operations?
One way is by practicing continuous delivery.
Okay. But how do we get better at
continuous delivery? Well, there are bunch of
technical practices that drive continuous delivery, things like
trunk base development or continuous integration.
The research program itself looks into about 30 different
capabilities like this whether it's truck-based development or
generative culture or documentation, lots and lots of
different capabilities. So I want to take you back to
that quick check. Not only will the Quick Check,
tell you, where do you sit? But it will also So make a
recommendation for you if you are a medium performer, here are
the three capabilities that are likely to have a big impact on
your team and then there's even further, follow-up questions
into each one of those capabilities to help you
prioritize but at the end of the day, the best thing to do is to
sit back and look at the body, all of the capabilities and have
an honest assessment of your team.
How are we doing with each one of these capabilities?
These what you're looking for, is the one that's holding you
back that constraint what's constraining, you focus your
improvement there. So that might mean, you need to
focus your improvement by improving your testing.
It might mean, you need to lean into something like blameless,
post-mortems. And what can you learn from
incidents? One of these things, maybe is
the thing that's holding you back, so make an investment in
improving in that area. And so together with the report
and the quick check, you can Really good insights into what
the various capabilities are and then decide, which is right for
your team. That finally, it depends on the
context of each team. So I really agree with that.
Speaking about a quick check again, make sure to try it out
because there will be recommendations as well.
The state of Devil's report itself covers a lot of these
technical aspect and if I may I add one more thing which is the
accelerate book written by the original authors of the Dora
report in the cold porcelain. So, in the accident book, it
also covers all the The fundamental basic important
technical practices, including also the cultural aspect which I
think is a good resource for people to get started.
Absolutely agree that accelerate book is is incredible.
You should absolutely read that. The other thing, I think while
we're on the topic on this site, you have access to all of the
state of devops reports. And just because the report says
2017 on, it does not mean that report is outdated, some of the
capabilities investigated. There are really important.
And I think, at the end of the day, Really what?
The door research program is trying to instill in all of us.
As an industry, is the practice and the mindset of continuous
Improvement. Let's design an experiment.
We believe that by getting better at truck-based
development. We will improve our software,
delivery operations, performance, and drive our
organizational performance. Let's test that theory and
improve our truck based development, and then reassess.
So let's measure again, we Find what's holding us back.
Make an improvement. We reassess so get into this
Loop in this practice of identifying where are we?
What's holding us back? Let's make a change to improve.
I love that the way you mentioned about continuous
Improvement aspect. It comes back to the lean and
agile principles and raised that way you need to also do
experiments test it out, any proof based on that findings.
Citing these feedback loop is really critical in devops lean
and agile. Mindset.
So, Nathan has been a really Pleasant conversation.
I learned a lot about the state of divorce report especially for
this year. So unfortunately we reach the
end of our conversation, but before I let you go, I have this
one. Normal question that I ask all
my guests which is to share your three technical leadership with
the. So think of it, like an advice
for the listeners hear from your experience.
What are the things that they should be thinking about?
Yes. All right, you freeze.
It also is technical leadership, so I'm going to start with some
really strong advice for leaders.
And that advice is this your team knows, what needs to be
done to improve, listen to them, ask them, what do we need to do
to improve? Give them audacious goals and
let them get to work. So your job as a leader is to
create that time and space and give them the resources that
they need to improve. Now, on the flip side of that,
I'll give some advice for the practitioners, the individual
contributors, you know what needs to be done.
Speak up. Share this with your leadership.
So drive that message forward consult with your team and learn
from each other and that lured from each other.
To me, that's the third and probably the most important each
and every one of us comes to work with a completely different
set of lived experience, a completely different set of
knowledge. It's really important that we
all share and learn from one another throw out.
The org chart throw out the structure.
Let's have those conversations. Ins make sure that everyone
feels included and like they belong as part of this team
because in truth, they do and we're better because of all of
us. And when we share our insights,
in our ideas, with one another side, we don't get my three.
Now, I must say it's a beautiful way of sharing about this
wisdom. It's all interrelated.
So the first is like, your team probably knows what to improve.
So, do ask the teeth and for the team members itself speak out,
actually, what to improve just speak up to your leaders and The
end we all learn from each other.
So really beautiful way of explaining this wisdom so
blatant for people who loves to connect more with you talking
more about devops report or anything related to devops or
Dora or maybe Google Cloud, when they can find you online.
Yes. The best place to find me is
Twitter and their I'm at Nathan Harvey.
I will just warn you that my father, misspelled my name or
chosen non-traditional spelling so it's n-ath.
Ey, that's my Twitter handle and then you can find me on LinkedIn
as well. So, search for my name, Nathan
Harvey with Annie, and you'll find me.
So, those are two good ways and Henry, I'm sure you can put some
links in the show notes there on how folks can reach me as well.
Definitely, I'll put it in the show notes.
So Nathan, thanks so much again for sharing your knowledge and
expertise year. Thank you so much.
Anyway, it's my pleasure. Thank you so much for having me,
and thank you for your dedication to Tech lead Journal.
I think we're going these types of insights and conversations to
the broader community. You're doing a real service to
everyone. Thank you so much.
It's been my pleasure. Thank you for listening to this
episode and for staying right till the end.
If you're highly enjoyed, please share it with your friends and
colleagues who you think would also benefit from listening to
this episode. And if you're new to the
podcast, make sure to subscribe and leave me your valuable
review and feedback. It really, really helps me a lot
in order to grow these podcasts better.
You can also find the full show notes of this conversation on
the episode page, at Tech Legion of the death website including
Doing the full transcript, interesting quotes, and links to
the resources and mentions from the conversation.
And lastly make sure to subscribe to the shows mailing
list on technology know the deaf to get notified for any future
episodes. Stay tuned for the next
technique Journal episode. And until then goodbye,