1
00:00:00,014 --> 00:00:02,138
There's this illusion that 
 
software should be easy to 

2
00:00:02,138 --> 00:00:04,050
create. 
And frankly, in the age of AI, 


3
00:00:04,058 --> 00:00:05,474
that illusion just gets even 
worse. 

4
00:00:05,704 --> 00:00:08,295
Today's guests are Elizabeth 
Hendrickson 
 and Joel Tosi, 

5
00:00:08,295 --> 00:00:10,764
co-authors of Signals and Levers
from IT Revolution. 

6
00:00:10,854 --> 00:00:13,225
Elizabeth is a veteran Silicon 
Valley 
 engineering executive, 

7
00:00:13,225 --> 00:00:15,808
while Joel has spent years 
coaching systems thinking 
 

8
00:00:15,816 --> 00:00:17,163
inside engineering teams 
worldwide. 

9
00:00:17,213 --> 00:00:19,653
The software in the organization
exists 
 inside of its own 

10
00:00:19,653 --> 00:00:21,893
system, but also exists inside 
of a larger ecosystem. 

11
00:00:21,903 --> 00:00:24,719
Everything is changing all the 
time, 
 right today can be wrong

12
00:00:24,719 --> 00:00:25,784
tomorrow. 
What is signal? 

13
00:00:25,784 --> 00:00:27,983
What is lever? 
Signals are simply quantifiable 

14
00:00:27,983 --> 00:00:30,413
or 
 qualitative measures of the
system. 

15
00:00:30,424 --> 00:00:33,303
Levers are the choices that you 
 can make in response to signals

16
00:00:33,354 --> 00:00:34,863
that is gonna influence the 
system. 

17
00:00:34,913 --> 00:00:36,889
Everyone in the system, 
regardless of 
 their role, 

18
00:00:36,889 --> 00:00:38,685
regardless of their positional 
authority, actually has more 

19
00:00:38,685 --> 00:00:40,503
levers at 
 their disposal than 
they think they do. 

20
00:00:40,683 --> 00:00:42,321
Tell us about these three 
illusions, 
 because sometimes 

21
00:00:42,321 --> 00:00:44,213
we are under curse living in 
this illusion. 

22
00:00:44,233 --> 00:00:46,785
The three illusions, illusions 
of 
 progress, illusions of 

23
00:00:46,785 --> 00:00:48,244
predictability, and illusions of
control. 

24
00:00:48,273 --> 00:00:50,761
The illusion of progress, we 
think 
 we're getting somewhere 

25
00:00:50,761 --> 00:00:52,654
quickly, but we- we're really 
not getting there. 

26
00:00:52,694 --> 00:00:55,166
And illusion of predictability, 
this 
 idea that we can predict 

27
00:00:55,166 --> 00:00:56,724
everything in software is just 
kind of silly. 

28
00:00:56,744 --> 00:00:58,333
And the last one, the illusion 
of control. 

29
00:00:58,354 --> 00:01:01,040
The idea that if we have 
meetings, 
 the problems will go

30
00:01:01,040 --> 00:01:02,573
away. 
If we have a risk control 
 

31
00:01:02,581 --> 00:01:05,251
board, we won't have defects. 
AI is like one big variable in 

32
00:01:05,251 --> 00:01:07,925
the 
 systems now, and most will
think AI will speed up their 

33
00:01:07,925 --> 00:01:09,354
development. 
So what are your point of view 

34
00:01:09,354 --> 00:01:11,132
in this? 
The question isn't, is this 

35
00:01:11,132 --> 00:01:12,473
useful or not? 
It's useful. 

36
00:01:12,533 --> 00:01:14,981
It's already proved itself. 
The question is, what is the 

37
00:01:14,981 --> 00:01:17,013
best 
 way for us to apply this 
technology? 

38
00:01:17,043 --> 00:01:19,489
Where AI is gonna end up 
falling, 
 if leadership doesn't

39
00:01:19,489 --> 00:01:21,336
understand the system at play, 
they could 
 introduce AI in 

40
00:01:21,336 --> 00:01:23,953
such a way that it exacerbates 
an existing problem. 

41
00:01:23,984 --> 00:01:26,248
One interesting thing that we 
leave as 
 an exercise for the 

42
00:01:26,248 --> 00:01:28,123
reader, what happens when you 
ship faster than you learn? 

43
00:01:44,341 --> 00:01:46,787
Hey, quick pause. 
My goal with Tech Lead Journal 

44
00:01:46,787 --> 00:01:49,371
is simple. 
Learn from the best in tech 
 so

45
00:01:49,371 --> 00:01:52,741
we can all grow together. 
If this resonates with you, hit 

46
00:01:52,741 --> 00:01:54,391
 subscribe to follow the 
channel. 

47
00:01:54,481 --> 00:01:57,367
It's the biggest way for you to 
support 
 the show and help us 

48
00:01:57,367 --> 00:01:59,401
keep bringing great guests and 
insights to you. 

49
00:01:59,941 --> 00:02:01,861
Thanks for being here, 
 and 
let's get back to it. 

50
00:02:03,167 --> 00:02:06,197
Welcome, Elisabeth and Joel, 
 
to Tech Lead Journal podcast. 

51
00:02:06,397 --> 00:02:08,227
Very excited to have both of you
here. 

52
00:02:09,137 --> 00:02:12,953
Both of them are the co-authors 
of the 
 new book, published by 

53
00:02:12,953 --> 00:02:15,386
IT Revolution. 
The title is called Signals & 

54
00:02:15,386 --> 00:02:18,091
Levers. 
So it's like one of those books,

55
00:02:18,091 --> 00:02:20,679

 that discuss about, you know, 
this kind of system thinking and

56
00:02:20,679 --> 00:02:22,947
all that. 
One of my favorite topics to 
 

57
00:02:22,955 --> 00:02:26,092
discuss because, every time I 
chat about it, there's always 
 

58
00:02:26,100 --> 00:02:27,847
something new that I learn 
somehow. 

59
00:02:28,397 --> 00:02:29,907
So welcome both of you to the 
show. 

60
00:02:30,067 --> 00:02:31,487
Looking forward for this 
conversation. 

61
00:02:32,442 --> 00:02:34,901
Thank you so much for having us.
Right. 

62
00:02:34,911 --> 00:02:37,461
So maybe in the beginning, let's
start 
 from the background 

63
00:02:37,461 --> 00:02:39,651
story, right? 
What made both of you 
 decide 

64
00:02:39,651 --> 00:02:42,867
to write this book? 
Sorry, Joel, go ahead. 

65
00:02:43,262 --> 00:02:45,052
All right, that's a wonderful 
story. 

66
00:02:46,362 --> 00:02:48,982
So a quick background for you, 
Henry. 

67
00:02:50,142 --> 00:02:53,266
I had been teaching a systems 
thinking 
 workshop, for some 

68
00:02:53,266 --> 00:02:55,742
clients, you know, for a few 
years, a little bit off and on. 

69
00:02:56,902 --> 00:03:01,218
And one day on… It's gonna sound
silly, 
 but one day on LinkedIn

70
00:03:01,218 --> 00:03:03,902
-- LinkedIn or Mastodon, 
Elisabeth you have to 
 keep me 

71
00:03:03,902 --> 00:03:06,742
honest -- Elisabeth had posted 
this video of this simulation 

72
00:03:06,742 --> 00:03:09,582
she 
 was working on, and it 
blew my mind. 

73
00:03:09,592 --> 00:03:11,874
Because when I was teaching the 
 systems thinking workshop, it 

74
00:03:11,874 --> 00:03:14,618
was a lot of hands-on kind of 
sticky 
 note stuff, and you 

75
00:03:14,618 --> 00:03:17,132
kind of had to kind of like feel
it out. 

76
00:03:17,502 --> 00:03:20,823
And then Elisabeth's simulation 
 allowed you to like make 

77
00:03:20,823 --> 00:03:24,664
decisions and see how the system
changed and, 
 you know, go 

78
00:03:24,664 --> 00:03:27,436
through it very rapidly. 
And I reached out to Elisabeth, 

79
00:03:27,436 --> 00:03:29,812
who 
 we had known each other 
for years, and I said, 

80
00:03:29,812 --> 00:03:33,117
"Elisabeth, this is 
 amazing! 
This is exactly what I've been 

81
00:03:33,117 --> 00:03:37,392
looking for forever. 
Can we 
 please work together on

82
00:03:37,392 --> 00:03:39,501
this?" 
And that's kind of how it's, it 

83
00:03:39,501 --> 00:03:41,059
started. 
Elisabeth, you, I know you have 

84
00:03:41,059 --> 00:03:44,370
 much more color to add to that.
Oh, and I was so grateful 
 when

85
00:03:44,370 --> 00:03:45,915
you reached out. 
Because that simulation, 
 I'm 

86
00:03:45,915 --> 00:03:48,485
still working on it. 
It's a passion project. 

87
00:03:49,005 --> 00:03:54,409
And I did the build a thing and 
then 
 figure out what to do 

88
00:03:54,409 --> 00:03:56,805
with it pattern that you're not 
supposed to do. 

89
00:03:56,865 --> 00:03:59,805
And I was so thrilled that 
anybody 
 had a real use for 

90
00:03:59,805 --> 00:04:03,385
this thing. 
But we started co-teaching and 

91
00:04:03,385 --> 00:04:06,627
ironically, 
 the simulation 
actually kind of fell by the 

92
00:04:06,627 --> 00:04:08,315
wayside the more that we 
co-taught. 

93
00:04:08,335 --> 00:04:10,562
But the more that we co-taught, 
 the more we realized, you know,

94
00:04:10,562 --> 00:04:13,653
this is really a book. 
And so that's how the book came 

95
00:04:13,653 --> 00:04:16,088
about. 
We've been working on it for 

96
00:04:16,088 --> 00:04:17,442
what? 
I think we started working on 
 

97
00:04:17,451 --> 00:04:20,285
it two and a half years ago. 
But three years ago, neither 
 

98
00:04:20,293 --> 00:04:24,325
one of us would've said that 
this was in the works at all. 

99
00:04:24,335 --> 00:04:27,227
Not on our bingo cards. 
Right. 

100
00:04:27,227 --> 00:04:29,597
Thank you for sharing such 
 an 
interesting story, right? 

101
00:04:29,597 --> 00:04:32,276
So I'm really curious about the 
simulation 
 because, yeah, 

102
00:04:32,276 --> 00:04:34,559
systems thinking is one thing 
that is quite difficult 
 

103
00:04:34,567 --> 00:04:36,177
actually, I think, to explain to
people. 

104
00:04:36,497 --> 00:04:39,541
Especially sometimes it's quite 
abstract 
 and, you know, there 

105
00:04:39,541 --> 00:04:41,967
are interrelated parts that you 
actually don't know about. 

106
00:04:42,227 --> 00:04:45,673
So yeah, looking forward to 
 
seeing the simulation being used

107
00:04:45,673 --> 00:04:50,332
in more wide audience. 
Before we continue, I want to 

108
00:04:50,332 --> 00:04:52,887
tell 
 you about our sponsor, 
Speechify AI. 

109
00:04:53,627 --> 00:04:55,747
Here's a problem you 
 might 
probably run into. 

110
00:04:56,247 --> 00:04:59,739
You need natural-sounding speech
in 
 your product, but every 

111
00:04:59,739 --> 00:05:02,993
text-to-speech option you test 
either sounds 
 robotic, costs a

112
00:05:02,993 --> 00:05:06,007
fortune at scale, or just isn't 
good enough to ship. 

113
00:05:06,727 --> 00:05:10,557
Speechify AI's Simba 3.2 
 
tackles both ends of that. 

114
00:05:11,047 --> 00:05:14,116
It's ranked number one on the 
 
Artificial Analysis blind 

115
00:05:14,116 --> 00:05:16,947
testing leaderboard for 
realness, and it's 
 priced 

116
00:05:16,947 --> 00:05:19,017
under $10 per million 
characters. 

117
00:05:19,737 --> 00:05:22,130
If you are building something 
more 
 conversational, they've 

118
00:05:22,130 --> 00:05:26,567
also got the Simba Voice Agents 
API, a low latency, 
 natural 

119
00:05:26,567 --> 00:05:29,595
sounding, built for the actual 
back-and-forth interaction 

120
00:05:29,595 --> 00:05:32,417
rather 
 than just reading text 
out loud. 

121
00:05:33,277 --> 00:05:36,403
Everything starts with a free 
tier 
 with no commitment, so 

122
00:05:36,403 --> 00:05:39,323
there's no real reason not to 
test it against 
 whatever 

123
00:05:39,323 --> 00:05:42,967
you're using right now. 
If voice is anywhere in your 
 

124
00:05:42,975 --> 00:05:46,187
product roadmap, head to 
speechify.ai to give it a try. 

125
00:05:46,637 --> 00:05:48,867
But make sure you don't 
 go to 
the wrong URL. 

126
00:05:49,167 --> 00:05:52,397
Here's the link one 
 more time,
speechify.ai. 

127
00:05:53,394 --> 00:05:55,434
And now let's get back to our 
episode. 

128
00:05:55,860 --> 00:05:59,798
So looking at the book, right, I
did 
 have a read before this 

129
00:05:59,798 --> 00:06:02,064
conversation. 
One of the very interesting 

130
00:06:02,064 --> 00:06:05,170
thing in 
 the introduction that
I pick is actually this story 

131
00:06:05,170 --> 00:06:07,662
about the software crisis. 
Why Maybe tell us a little bit 


132
00:06:07,670 --> 00:06:11,142
about this because I think it's 
worth to re-emphasize about 

133
00:06:11,142 --> 00:06:14,245
this. 
Yeah, absolutely. 

134
00:06:14,245 --> 00:06:19,689
Especially because it puts all 

of our experiences in context. 

135
00:06:20,079 --> 00:06:23,029
So the year was, I believe it 
was 1968. 

136
00:06:23,079 --> 00:06:25,519
I'm not looking at my cheat 
 
sheets, and I'm not great at 

137
00:06:25,519 --> 00:06:32,299
dates, but it was late '60s. 
And NATO, of all organizations, 

138
00:06:32,299 --> 00:06:36,209
organized 
 a software 
engineering conference to 

139
00:06:36,219 --> 00:06:40,119
address the emerging software 
crisis. 

140
00:06:40,979 --> 00:06:45,507
So at the time, the field of 
computing 
 was so new, and yet,

141
00:06:45,507 --> 00:06:48,381
as I was reading through this 
historical document, 'cause 
 

142
00:06:48,389 --> 00:06:50,749
the notes are widely available 
on the web. 

143
00:06:50,759 --> 00:06:53,387
You can search for the NATO 
 
conference on software 

144
00:06:53,387 --> 00:06:55,959
engineering. 
And parts of the notes that came

145
00:06:55,959 --> 00:06:57,999
out of 
 that conference read 
like a transcript. 

146
00:06:57,999 --> 00:07:01,399
It's like the back and forth 
 
he, well, it was all hes. 

147
00:07:01,409 --> 00:07:05,089
So he said, he said, he said. 
And I'm reading this and I'm 

148
00:07:05,089 --> 00:07:07,000
like, "I 
 have been in that 
meeting." 

149
00:07:07,000 --> 00:07:11,419
I mean, I … This feels so 
familiar. 

150
00:07:11,419 --> 00:07:15,336
And they're all complaining 
about how, 
 Some of them are 

151
00:07:15,336 --> 00:07:18,789
complaining that software takes 
too long and costs too much 

152
00:07:18,789 --> 00:07:22,109
money, 
 and others who clearly 
were a little closer to the 

153
00:07:22,109 --> 00:07:23,789
keyboard were saying, "Well, 
 
you don't understand. 

154
00:07:23,789 --> 00:07:26,771
How hard it is, and this is 
why." 

155
00:07:26,771 --> 00:07:31,484
And it felt so familiar. 
The only thing that was 

156
00:07:31,484 --> 00:07:34,299
different 
 was the vocabulary 
around the specific 

157
00:07:34,299 --> 00:07:37,347
technologies. 
But the essence of the 
 meeting

158
00:07:37,347 --> 00:07:40,153
was the same. 
So that is the meeting that 
 

159
00:07:40,161 --> 00:07:43,409
gave us the, that popularized 
the term the software crisis. 

160
00:07:43,999 --> 00:07:47,478
And our realization as we were 
working 
 through the materials 

161
00:07:47,478 --> 00:07:51,748
for this book was that this is--
Jerry Weinberg liked 
 to say 

162
00:07:51,748 --> 00:07:55,699
that it looks like a crisis, but
it's the end of an illusion. 

163
00:07:56,019 --> 00:07:57,904
He wrote that in one of his 
books, 
 and he also liked to 

164
00:07:57,904 --> 00:08:00,589
say it a lot in person, and it 
is so true. 

165
00:08:00,959 --> 00:08:03,515
There's this illusion that 
 
software should be easy to 

166
00:08:03,515 --> 00:08:06,322
create. 
And frankly, in the age of AI, 

167
00:08:06,322 --> 00:08:09,387
that 
 illusion just gets even 
worse, because obviously the 

168
00:08:09,387 --> 00:08:12,375
magic genie is gonna 
 create 
all the software for us, so we 

169
00:08:12,375 --> 00:08:15,444
don't need to worry about all of

 these, you know, hard things 

170
00:08:15,444 --> 00:08:17,299
about software creation anymore,
right? 

171
00:08:17,309 --> 00:08:21,773
No, that's not how that works. 
So yes, we have been, as an 

172
00:08:21,773 --> 00:08:24,249
industry, 
 dealing with the 
supposed software crisis, that 

173
00:08:24,249 --> 00:08:27,291
the fact that it takes so 
 
long, longer than anybody 

174
00:08:27,291 --> 00:08:31,121
expects to create it, and it's 
harder and costs more 
 since 

175
00:08:31,121 --> 00:08:35,167
the very beginning of the field.
And this is not a problem that's

176
00:08:35,167 --> 00:08:38,385
going 
 away anytime soon, no 
matter what kind of advances we 

177
00:08:38,385 --> 00:08:42,547
have in our field. 
Yeah, so it's actually very 
 

178
00:08:42,554 --> 00:08:43,837
interesting when you mentioned 
that. 

179
00:08:43,876 --> 00:08:47,363
I'm sure we are, we have been in
that 
 kind of meeting as well, 

180
00:08:47,363 --> 00:08:50,377
where we talk about, you know, 
first is schedule, right? 

181
00:08:50,447 --> 00:08:53,917
We cannot meet the deadline. 
Second thing is the cost 
 to 

182
00:08:53,917 --> 00:08:56,382
deliver the software. 
And then the complexity, how 

183
00:08:56,382 --> 00:08:58,737
hard it 
 is to actually change 
the software. 

184
00:08:59,317 --> 00:09:01,857
I think in a sense also, it's 
 
kind of like relative, right? 

185
00:09:01,857 --> 00:09:04,011
Because technology changes, 
there are 
 some things that 

186
00:09:04,011 --> 00:09:07,267
gets easier, but still these 
kind of things, this kind of 

187
00:09:07,267 --> 00:09:09,437
theme 
 keep appearing, over and
over again. 

188
00:09:09,447 --> 00:09:13,607
Why do you think it's never been
a solved 
 problem, or we just 

189
00:09:13,607 --> 00:09:16,459
don't appreciate that things 
actually have advanced since 

190
00:09:16,459 --> 00:09:19,369
then? 
Well, I mean, I agree. 

191
00:09:19,369 --> 00:09:22,979
Some things get easier and, but 
 expectations also get higher 

192
00:09:22,979 --> 00:09:24,749
about what we should be able to 
do. 

193
00:09:25,279 --> 00:09:29,111
So some of the technology that I
use 
 on a day-to-day basis 

194
00:09:29,111 --> 00:09:31,759
would've been science fiction 
when I was a child. 

195
00:09:32,149 --> 00:09:34,469
So certainly things have 
advanced. 

196
00:09:34,959 --> 00:09:39,121
I think it really does boil down
to the 
 fact that it's a system

197
00:09:39,121 --> 00:09:43,117
as a whole, and that the, that 
maybe our expectations 
 for how

198
00:09:43,117 --> 00:09:47,589
much it should cost or how long 
it should take are unrealistic. 

199
00:09:47,619 --> 00:09:50,039
That's absolutely true in many 
cases. 

200
00:09:50,379 --> 00:09:54,749
But also there are complications

 that we can't predict by 

201
00:09:54,769 --> 00:10:00,358
thinking linearly about them. 
The, everything that we 
 do 

202
00:10:00,358 --> 00:10:03,553
affects the system. 
And I, we like to say in the 

203
00:10:03,553 --> 00:10:05,969
book, "You 
 poke the system and
the system pokes back." 

204
00:10:05,969 --> 00:10:11,489
So when you do a thing, it may 

have some very surprising 

205
00:10:11,489 --> 00:10:15,710
consequences. 
And even beyond that, it's, 

206
00:10:15,710 --> 00:10:19,247
your, 
 the software and the 
organization exists inside of 

207
00:10:19,247 --> 00:10:21,792
its own system, 
 but also 
exists inside of a larger 

208
00:10:21,822 --> 00:10:24,554
ecosystem of its, all of its 
 
competitors and the world around

209
00:10:24,554 --> 00:10:27,210
it. 
So like, there is no, we'll just

210
00:10:27,210 --> 00:10:31,156
work 
 on this for six months 
and the world won't change, and 

211
00:10:31,156 --> 00:10:33,712
our competitors won't 
 change, 
and the customers won't change. 

212
00:10:33,762 --> 00:10:35,662
Everything is changing all the 
time. 

213
00:10:36,112 --> 00:10:38,390
And so, and another thing I like

 to say in the book is, you 

214
00:10:38,390 --> 00:10:40,402
know, right today can be wrong 
tomorrow. 

215
00:10:40,532 --> 00:10:43,959
And I think that's why this why 
it 
 won't be a solved problem, 

216
00:10:43,959 --> 00:10:46,846
'cause we could have the best 
ideas and 
 the best approaches 

217
00:10:46,846 --> 00:10:49,618
today, but then everything 
around us, the system 
 around 

218
00:10:49,618 --> 00:10:53,740
us changes, and now we're wrong.
And so we have to be able 
 to 

219
00:10:53,740 --> 00:10:56,047
sense those things. 
Yeah. 

220
00:10:56,467 --> 00:10:58,977
So I think we all know that AI 
is coming. 

221
00:10:59,017 --> 00:11:02,221
I'm sure this thing will still 

again a crisis in the next few 

222
00:11:02,221 --> 00:11:04,997
years ahead, even though the AI 
maybe 
 helps us to generate a 

223
00:11:04,997 --> 00:11:08,110
lot of code. 
So which brings me to the thing 

224
00:11:08,110 --> 00:11:10,257
that you 
 mentioned in the 
intro as well, right? 

225
00:11:10,287 --> 00:11:12,333
You mentioned that software 
delivery, 
 software 

226
00:11:12,333 --> 00:11:15,895
development, right, is actually,
an adaptive sociotechnical 

227
00:11:15,895 --> 00:11:19,472
system. 
I think for many, maybe I would 

228
00:11:19,472 --> 00:11:22,278
say 
 non-technical users, this 
is something that is quite 

229
00:11:22,278 --> 00:11:24,094
important for them to 
 
understand, especially if they 

230
00:11:24,094 --> 00:11:26,417
want to relate later to the 
systems thinking. 

231
00:11:26,417 --> 00:11:29,327
So tell us about this adaptive 

sociotechnical system. 

232
00:11:30,542 --> 00:11:33,674
Sure. 
So we'll start with the kind 
 

233
00:11:33,682 --> 00:11:37,152
of simpler, more evident parts, 
the socio aspect of it. 

234
00:11:37,512 --> 00:11:41,188
Software is built by people, 
interacting 
 with people, for 

235
00:11:41,188 --> 00:11:43,808
use by people. 
Even if machines end up using 

236
00:11:43,808 --> 00:11:45,432
it, 
 it's for the benefit of 
people. 

237
00:11:45,442 --> 00:11:48,712
So there's a definitely a social

 aspect, socio aspect of that. 

238
00:11:49,512 --> 00:11:52,652
People being complex creatures, 
 that of course just adds to the

239
00:11:52,732 --> 00:11:57,910
unpredictability of the system. 
The adaptability is very much 
 

240
00:11:57,918 --> 00:12:02,132
that the outputs from the system
becomes feedbacks into the 

241
00:12:02,132 --> 00:12:03,684
system. 
So the system's constantly 

242
00:12:03,684 --> 00:12:06,916
adapting to 
 the situations 
that's fed into it and all of 

243
00:12:06,916 --> 00:12:10,807
the influences with inside it. 
The fact that it's a system has 

244
00:12:10,807 --> 00:12:12,082
 a lot of interesting 
attributes. 

245
00:12:12,232 --> 00:12:14,672
So when we talk about a system, 
 we talk about a system of 

246
00:12:14,672 --> 00:12:19,854
software delivery as a whole. 
So when we think about it as a 

247
00:12:19,854 --> 00:12:23,488
system, 
 you have a system of 
your team, and your team is 

248
00:12:23,488 --> 00:12:26,894
working on its product. 
But your team exists inside of 


249
00:12:26,902 --> 00:12:29,216
a larger system, inside of your 
organization, possibly an 

250
00:12:29,216 --> 00:12:32,522
organization 
 or a portfolio 
which exists inside of a larger 

251
00:12:32,522 --> 00:12:34,768
system, which is the whole 
 
company, which exists inside of 

252
00:12:34,768 --> 00:12:37,492
a larger system, which is the, 
you know, 
 economy or the 

253
00:12:37,492 --> 00:12:40,636
product space it's in. 
And so you have this interesting

254
00:12:40,636 --> 00:12:42,612

 thing where it's systems all 
the way up and down. 

255
00:12:43,062 --> 00:12:46,232
And that because it's a system 

and it's adaptive, that means 

256
00:12:46,232 --> 00:12:49,370
that they're all influencing and

 affecting each other 

257
00:12:49,370 --> 00:12:52,109
constantly. 
So the decision that the 

258
00:12:52,109 --> 00:12:55,774
organization 
 makes affects the
team, but it also affects the 

259
00:12:55,774 --> 00:12:57,794
broader economy. 
If the organization decides to 


260
00:12:57,802 --> 00:13:00,542
adopt a new technology, it 
affects the teams directly, and 

261
00:13:00,542 --> 00:13:02,162
of course, 
 affects the, its 
competitors. 

262
00:13:02,602 --> 00:13:04,912
And so you see all of this 
 
kind of adaptive situations 

263
00:13:04,912 --> 00:13:08,682
happening inside the world. 
And then the other interesting 

264
00:13:08,682 --> 00:13:13,538
aspect 
 about it being a system
is that, no change happens in 

265
00:13:13,538 --> 00:13:15,546
isolation. 
And I think that's super 
 

266
00:13:15,554 --> 00:13:18,112
interesting, especially in 
software, because everybody's 

267
00:13:18,112 --> 00:13:21,022
been having 
 the best 
intentions, right? 

268
00:13:21,022 --> 00:13:23,736
We've seen the DevOps teams, and

 we've seen the platform teams,

269
00:13:23,736 --> 00:13:28,216
and we've seen microservice, and
all of 
 these were great ideas 

270
00:13:28,216 --> 00:13:30,987
in isolation. 
And they're great ideas in 

271
00:13:30,987 --> 00:13:33,650
general, 
 but when you take 
them, you have to take them in 

272
00:13:33,650 --> 00:13:35,422
the context of 
 the system it 
puts into play. 

273
00:13:35,562 --> 00:13:38,277
So for example, if you're 
saying, well, 
 we have a 

274
00:13:38,277 --> 00:13:40,523
monolith, and it's too hard 
because everybody's always 

275
00:13:40,523 --> 00:13:44,320
stepping on 
 each other's toes,
therefore we're gonna go to a 

276
00:13:44,320 --> 00:13:48,676
microservices architecture. 
It's not just that the team gets

277
00:13:48,676 --> 00:13:50,902
impacted. 
It's now the deployments, and 
 

278
00:13:50,910 --> 00:13:53,247
operations, and support, and 
resiliency, and the product 

279
00:13:53,247 --> 00:13:55,272
changes, and all 
 these, it has
this ripple effect. 

280
00:13:55,702 --> 00:13:58,002
And so you can't make a 
 change
in isolation without 

281
00:13:58,002 --> 00:14:01,918
understanding the system play. 
Elisabeth, what would you add to

282
00:14:01,918 --> 00:14:05,607
that? 
I would say that that's right 

283
00:14:05,607 --> 00:14:08,509
on. 
And I think we saw an example of

284
00:14:08,509 --> 00:14:12,495
this 
 two decades ago when 
Agile was the hot thing and 

285
00:14:12,495 --> 00:14:16,834
teams started adopting Agile. 
We saw how a team adopt Agile, 

286
00:14:16,834 --> 00:14:20,647
and 
 all of a sudden now we see
the ripple effect pushing out 

287
00:14:20,647 --> 00:14:23,699
from the team. 
So Joel, you described the 

288
00:14:23,699 --> 00:14:26,823
organization 
 does a thing, it 
affects the teams, and it 

289
00:14:26,823 --> 00:14:29,901
affects their competitors. 
So that's a system in this 
 

290
00:14:29,909 --> 00:14:33,671
sort of web of systems. 
That happens everywhere. 

291
00:14:34,031 --> 00:14:40,521
A team adopting a new practice, 
 a team adopts AI today, that's 

292
00:14:40,521 --> 00:14:42,881
gonna have a ripple effect on 
 
the rest of the organization. 

293
00:14:43,201 --> 00:14:47,601
So coming back to your question,
Henry, 
 you asked, so adaptive 

294
00:14:47,601 --> 00:14:49,931
sociotechnical system, help me 
unpack that. 

295
00:14:50,331 --> 00:14:56,677
It's adaptive because the output

 of the system then becomes an 

296
00:14:56,677 --> 00:14:59,393
input into the system. 
So the system is constantly 

297
00:14:59,393 --> 00:15:02,288
adapting. 
The system is made of people for

298
00:15:02,288 --> 00:15:04,881

 people, and technology is at 
the core. 

299
00:15:05,271 --> 00:15:08,741
And that is why it is so 
 
complex to reason about. 

300
00:15:10,297 --> 00:15:11,817
Yeah, so thanks for unpacking 
that. 

301
00:15:11,817 --> 00:15:13,477
Definitely, quite interesting, 
right? 

302
00:15:13,477 --> 00:15:16,795
So I think mostly the people who
work 
 in tech really understand

303
00:15:16,795 --> 00:15:18,732
this, right? 
Because there are so many 

304
00:15:18,732 --> 00:15:21,768
variables, 
 there are so many 
things that could affect, you 

305
00:15:21,768 --> 00:15:23,967
know, the 
 velocity of software
development. 

306
00:15:24,267 --> 00:15:27,058
But I think when we talk about, 
you 
 know, delivering software 

307
00:15:27,058 --> 00:15:30,227
to the business or the 
executives, all they do is just 

308
00:15:30,227 --> 00:15:31,727
 create like, for example, 
deadlines. 

309
00:15:31,727 --> 00:15:35,019
You know, pressure us, you know.
"Okay, we have to deliver on 

310
00:15:35,019 --> 00:15:37,301
this date." 
 
Maybe throw more people, throw 

311
00:15:37,301 --> 00:15:39,757
more budget for tools like AI, 
for example. 

312
00:15:40,077 --> 00:15:45,031
So tell us why seems like the 
other counterparts of the 

313
00:15:45,031 --> 00:15:47,727
technology people 
 actually 
doesn't seem to get it. 

314
00:15:49,063 --> 00:15:51,780
Well, first of all, let's be 
fair and 
 acknowledge that 

315
00:15:51,780 --> 00:15:55,853
sometimes that works. 
And sometimes it does. 

316
00:15:56,163 --> 00:15:58,883
And sometimes the people… 
 
Like, let's take pressure. 

317
00:15:59,563 --> 00:16:02,411
I've, I have had leaders tell… 
When 
 I was an engineering 

318
00:16:02,411 --> 00:16:06,031
leader, I had a business leader 
say to me, "Oh, come 
 on, 

319
00:16:06,031 --> 00:16:09,740
Elisabeth, you know, you have to
light a fire under engineers to 

320
00:16:09,740 --> 00:16:12,757
get 
 them to do anything." 
This person was very clearly 

321
00:16:12,757 --> 00:16:16,127
used to pressure 
 working, and 
it didn't work with me, but 

322
00:16:16,127 --> 00:16:19,064
that's 'cause I'm extra 
stubborn, 
 which is why I work 

323
00:16:19,064 --> 00:16:22,516
for myself now. 
But he was accustomed to it 

324
00:16:22,516 --> 00:16:26,993
working, 
 but also he hadn't 
been in a job for more than, 

325
00:16:26,993 --> 00:16:30,207
like, two years. 
He would… Even if he stayed with

326
00:16:30,207 --> 00:16:32,700
the 
 same company, this 
particular person was 

327
00:16:32,700 --> 00:16:34,983
constantly… he was very 
ambitious. 

328
00:16:35,013 --> 00:16:37,285
Nothing wrong with being 
ambitious, but he 
 was 

329
00:16:37,285 --> 00:16:40,127
constantly shifting roles, which
means that when the chickens 

330
00:16:40,127 --> 00:16:43,373
came home to roost 
 and all of 
that pressure led to technical 

331
00:16:43,373 --> 00:16:46,766
debt, which led to slower 
delivery, 
 he wasn't there to 

332
00:16:46,766 --> 00:16:50,423
feel the pain. 
And so let's start with 
 why do

333
00:16:50,423 --> 00:16:53,213
they pressure? 
Because it works. 

334
00:16:53,863 --> 00:16:58,174
But what they don't have to live

 with is the consequences of 

335
00:16:58,174 --> 00:17:00,947
that. 
And I think that we're gonna 
 

336
00:17:00,955 --> 00:17:06,269
see that even more with AI. 
AI AI is… frankly, it's magical.

337
00:17:06,269 --> 00:17:09,919
It is science fiction in its 
 
ability to deliver not just 

338
00:17:09,919 --> 00:17:13,529
software, but good software. 
And there are a lot of folks who

339
00:17:13,529 --> 00:17:16,913
were 
 working with the tools, 
like six months or a year ago, 

340
00:17:16,913 --> 00:17:20,153
and experiencing the 
 
frustrations of the AI lying to 

341
00:17:20,153 --> 00:17:22,035
them. 
And I'm not saying it doesn't 

342
00:17:22,035 --> 00:17:24,961
still 
 do that, but I've been 
using… i've been working with 

343
00:17:24,961 --> 00:17:27,313
Claude, partnering 
 with Claude
on this simulation, which 

344
00:17:27,313 --> 00:17:28,673
doesn't show up in the book by 
the way. 

345
00:17:28,673 --> 00:17:31,797
But on this simulation, Claude 
and I have 
 been partnering 

346
00:17:31,797 --> 00:17:34,933
closely, and Claude's ability to
write good code is amazing. 

347
00:17:35,043 --> 00:17:38,856
And yet, if I were not paying 
 
very close attention to the 

348
00:17:38,856 --> 00:17:42,356
design decisions that Claude is 
making, 
 then I would be in a 

349
00:17:42,356 --> 00:17:45,041
world of hurt. 
And so the pressure is gonna 

350
00:17:45,041 --> 00:17:46,763
lead 
 people to say, "Yeah, 
whatever. 

351
00:17:46,763 --> 00:17:48,493
It's good enough. 
It, clearly it works. 

352
00:17:48,493 --> 00:17:51,941
It's fine." 
And then a year from 
 now, how 

353
00:17:51,941 --> 00:17:56,173
many of those systems are gonna 
be so convoluted that 
 they're 

354
00:17:56,173 --> 00:18:00,229
impossible to reason about? 
So I… You know, why do they not 

355
00:18:00,229 --> 00:18:01,793
get it? 
'Cause they don't have to. 

356
00:18:02,953 --> 00:18:05,259
They're not gonna be the ones 
who actually 
 have to live with

357
00:18:05,259 --> 00:18:09,044
the consequences. 
I think I might even 
 give a 

358
00:18:09,044 --> 00:18:15,151
little, yes and. 
Because some managers and some 


359
00:18:15,159 --> 00:18:20,056
executives are there for a bit. 
And so what I would even offer 

360
00:18:20,056 --> 00:18:24,574
up for 
 the audience, Henry, is
that, if were to take like the 

361
00:18:24,574 --> 00:18:27,086
best intentions, I 
 think 
they're all doing the best with 

362
00:18:27,086 --> 00:18:28,976
the information that they have, 
right? 

363
00:18:28,976 --> 00:18:32,028
So like if I were to kinda say, 
"Well, 
 why is a manager 

364
00:18:32,028 --> 00:18:34,080
applying pressure?" 
Well, it's probably because 

365
00:18:34,080 --> 00:18:36,456
they're 
 feeling a 
responsibility to deliver 

366
00:18:36,456 --> 00:18:39,316
something to a customer because 
 of some other kind of promises.

367
00:18:39,876 --> 00:18:43,008
And they might not understand 
all of 
 the other things, all 

368
00:18:43,008 --> 00:18:45,949
the systemic effects that are 
influencing the 
 ability to 

369
00:18:45,949 --> 00:18:48,647
ship the software. 
And so the only thing, like in 

370
00:18:48,647 --> 00:18:50,759
the 
 book we call them levers, 
the only levers that they're 

371
00:18:50,759 --> 00:18:53,390
able to reach, the 
 only levers
that they know that they have 

372
00:18:53,390 --> 00:18:57,774
available to them, are pressure.
And so, I think they're doing 
 

373
00:18:57,782 --> 00:19:01,076
the best what they can with 
maybe a limited perspective. 

374
00:19:02,256 --> 00:19:05,996
And they just don't know 
different, right? 

375
00:19:05,996 --> 00:19:09,386
And I think that's one of the 
big 
 intentions of the book was

376
00:19:09,386 --> 00:19:12,120
to help executives and help 
leadership 
 understand like 

377
00:19:12,120 --> 00:19:15,492
it's a big world. 
There's a lot of things we need 

378
00:19:15,492 --> 00:19:18,764
to 
 understand because it's not
linear. 100%. 

379
00:19:18,764 --> 00:19:22,225
And I didn't mean to imply that,
you 
 know, the people who are 

380
00:19:22,225 --> 00:19:24,347
putting pressure are evil in any
way, shape or form. 

381
00:19:24,677 --> 00:19:28,619
They're doing the best they can 
to 
 achieve their objectives 

382
00:19:28,619 --> 00:19:31,087
with the levers that they have 
available. 

383
00:19:32,857 --> 00:19:35,567
And for that matter, they are 
 
probably under pressure as well.

384
00:19:35,827 --> 00:19:37,971
The business leader who I was 
describing 
 who said, "You 

385
00:19:37,971 --> 00:19:40,819
know, Elisabeth, you have to 
light a fire under engineers," 

386
00:19:40,819 --> 00:19:43,137
he 
 was under enormous amounts 
of pressure. 

387
00:19:43,157 --> 00:19:46,107
And so he's responding. 
That's, you know, that's 
 the 

388
00:19:46,107 --> 00:19:49,247
adaptive part. 
He's responding to the forces 

389
00:19:49,247 --> 00:19:54,687
that are 
 acting on him and, 
you know, then he's turning 

390
00:19:54,687 --> 00:19:58,549
around and passing that on. 
But to Joel, Joel is hinting at 

391
00:19:58,549 --> 00:20:00,757
the fact 
 that there are other 
levers available. 

392
00:20:03,261 --> 00:20:06,271
Since both of you have 
 
mentioned about levers, right? 

393
00:20:06,271 --> 00:20:08,971
So I think this, probably 
signals and 
 levers, right, 

394
00:20:08,971 --> 00:20:12,232
referring to the title of the 
book, are the two things that 
 

395
00:20:12,240 --> 00:20:15,120
we engineering leaders or maybe 
all the people within the 

396
00:20:15,120 --> 00:20:19,109
systems, right, are able 
 to 
observe and play around with in 

397
00:20:19,109 --> 00:20:21,361
order to make better changes to 
the system. 

398
00:20:21,361 --> 00:20:24,865
So maybe explain a little bit 
about what 
 is signal, what is 

399
00:20:24,865 --> 00:20:28,438
lever for all of us. 
Sure. 

400
00:20:29,578 --> 00:20:33,655
When we talk about signals, 
 
signals are simply quantifiable 

401
00:20:33,655 --> 00:20:36,138
or qualitative measures of the 
system. 

402
00:20:36,158 --> 00:20:39,406
Things you can either 
experience, 
 measure, sometimes

403
00:20:39,406 --> 00:20:42,883
you just feel. 
So like obvious signals that a 

404
00:20:42,883 --> 00:20:47,254
lot 
 of teams use and a lot of 
management might use are things 

405
00:20:47,254 --> 00:20:51,544
like cycle 
 time or defect rate
or ship or deployment rate or 

406
00:20:51,544 --> 00:20:55,786
batch size. 
Like these are all kind of 
 

407
00:20:55,794 --> 00:20:58,198
very quantitative signals that 
we could all see. 

408
00:20:58,608 --> 00:21:01,340
But there's also all of these 
 
kind of qualitative ones as 

409
00:21:01,340 --> 00:21:03,328
well. 
The pressure the team feels. 

410
00:21:03,338 --> 00:21:06,672
Sometimes, some people in your 

audience may love meetings. 

411
00:21:06,892 --> 00:21:10,529
But let's just be honest, many 

times meetings are a signal and 

412
00:21:10,529 --> 00:21:12,212
we don't think about them as a 
signal. 

413
00:21:12,222 --> 00:21:14,910
Sometimes they're a signal that 
nobody 
 really understands 

414
00:21:14,910 --> 00:21:18,220
what's going on, or it's a 
signal we're doing too much, or 

415
00:21:18,220 --> 00:21:20,650
 it's a signal that there's a 
different problem because we're 

416
00:21:20,650 --> 00:21:23,247
having the same 
 meeting over 
and over again, and we think 

417
00:21:23,247 --> 00:21:26,517
that the meeting's the answer. 
So there's all, there's this 

418
00:21:26,517 --> 00:21:29,878
wonderful 
 world of signals, 
and one of the largest intents 

419
00:21:29,878 --> 00:21:35,147
of the book was to make the 
 
audience aware of the myriad of 

420
00:21:35,147 --> 00:21:38,152
signals that are around them so 
they can make 
 better sense of 

421
00:21:38,152 --> 00:21:40,462
what's happening. 
I did signals. 

422
00:21:40,462 --> 00:21:42,222
How about you take on levers, 
Elisabeth? 

423
00:21:42,551 --> 00:21:44,211
Sure. 
Well, levers are the choices 

424
00:21:44,211 --> 00:21:46,691
that 
 you can make in response 
to signals. 

425
00:21:47,181 --> 00:21:50,500
And so by recognizing that it is

 an adaptive sociotechnical 

426
00:21:50,500 --> 00:21:54,327
system, that helps you think 
about the fact 
 that lever that

427
00:21:54,327 --> 00:21:58,551
looks so obvious. 
Like the velocity is a signal. 

428
00:21:58,781 --> 00:22:00,755
I'm gonna come back to that 
signal in a 
 minute 'cause it's

429
00:22:00,755 --> 00:22:03,791
not a great signal. 
But, you know, velocity is a 

430
00:22:03,791 --> 00:22:06,814
signal 
 that a lot of 
organizations use, and the 

431
00:22:06,814 --> 00:22:09,697
velocity is lower than we want, 
so 
 we're gonna pull the 

432
00:22:09,697 --> 00:22:13,171
pressure lever. 
That's an example of a choice. 

433
00:22:13,431 --> 00:22:17,415
But it turns out that levers can
be big, 
 like big policy 

434
00:22:17,415 --> 00:22:19,869
levers, like we're gonna 
reorganize or we're gonna 

435
00:22:19,869 --> 00:22:23,736
embrace this new 
 tool or we're
gonna do this big rollout, but 

436
00:22:23,736 --> 00:22:28,141
they can also be really small. 
So there's two categories, two 


437
00:22:28,149 --> 00:22:31,935
giant categories of levers. 
There's policy and then there's 

438
00:22:31,935 --> 00:22:34,663
culture. 
Policy is anything that you need

439
00:22:34,663 --> 00:22:38,191

 organizational authority to 
enact, like a reorg or changing 

440
00:22:38,191 --> 00:22:42,231
the hiring criteria 
 or… So 
those are all potential levers. 

441
00:22:43,061 --> 00:22:47,229
But also culture, literally 
everyone 
 in the organization 

442
00:22:47,229 --> 00:22:50,823
is pulling culture levers all 
the time, 
 whether or not they 

443
00:22:50,823 --> 00:22:54,078
realize it. 
Just deciding whether or not to 

444
00:22:54,078 --> 00:22:58,385
 attend that meeting is an 
example of a lever that everyone

445
00:22:58,385 --> 00:23:01,743
gets to pull. 
And so if you just think about 

446
00:23:01,743 --> 00:23:05,001
that 
 one decision, like you're
thinking to yourself, "Okay, 

447
00:23:05,001 --> 00:23:11,699
this is a Zoom 
 meeting that is
a ceremony with 73 people 

448
00:23:11,699 --> 00:23:16,571
invited, of whom only 
 
30-something are gonna attend. 

449
00:23:16,581 --> 00:23:19,836
And of the 30-something, half of

 them are gonna have their 

450
00:23:19,836 --> 00:23:22,071
cameras off and be completely 
disengaged. 

451
00:23:22,121 --> 00:23:25,301
And then partway through, 
 
somebody's gonna say, 'Hang on. 

452
00:23:25,601 --> 00:23:27,191
could you repeat the question?'"
Right? 

453
00:23:27,531 --> 00:23:29,761
You- We've all been in 
 this 
meeting, right? 

454
00:23:30,331 --> 00:23:34,141
And you get to decide, "Should I

 bother to attend?" 

455
00:23:34,141 --> 00:23:36,781
That decision alone, that is an 
example of a lever. 

456
00:23:36,781 --> 00:23:41,053
So a lever is every choice that 
you're 
 making that is gonna 

457
00:23:41,053 --> 00:23:44,361
influence the system. 
And spoiler, everyone in the 

458
00:23:44,361 --> 00:23:47,629
system, 
 regardless of their 
role, regardless of their 

459
00:23:47,629 --> 00:23:50,301
positional authority, 
 actually
has more levers at their 

460
00:23:50,301 --> 00:23:51,761
disposal than they think they 
do. 

461
00:23:53,128 --> 00:23:56,638
Yeah, so I think when I read 
that part, 
 right, so when you 

462
00:23:56,638 --> 00:23:59,892
mention levers can be two 
categories, policy and culture, 

463
00:23:59,892 --> 00:24:02,024
right? 
I think it's quite interesting, 

464
00:24:02,024 --> 00:24:05,170
 right, to me especially. 
Because one thing that we always

465
00:24:05,170 --> 00:24:08,606
realize 
 is that whenever we 
wanna make a change in, let's 

466
00:24:08,606 --> 00:24:11,835
say, engineering organization, 

right, we make policy, new 

467
00:24:11,835 --> 00:24:14,483
policy. 
Maybe new executives come in and

468
00:24:14,483 --> 00:24:18,140
decide 
 a new policy, but we 
forgot about the culture part, 

469
00:24:18,140 --> 00:24:21,613
the aspects where, you know, 
 
like how things work, how people

470
00:24:21,613 --> 00:24:23,548
feel. 
I think that's also another 

471
00:24:23,548 --> 00:24:26,460
lever that 
 not just the 
leaders can do, can take, can 

472
00:24:26,460 --> 00:24:30,376
pull, but also everyone in the, 
you 
 know, in the team or in 

473
00:24:30,376 --> 00:24:33,728
the organization. 
So you mentioned about velocity.

474
00:24:33,728 --> 00:24:36,138
I think you might refer 
 that 
as a proxy metric. 

475
00:24:36,138 --> 00:24:39,163
So let's go into that, like 
because 
 in so many different 

476
00:24:39,163 --> 00:24:41,624
organizations still, this kind 
of proxy metrics is 
 something 

477
00:24:41,624 --> 00:24:44,538
that is quite dangerous. 
Maybe like velocity, lines of 

478
00:24:44,538 --> 00:24:47,758
code even, 
 number of 
incidents, those kind of stuff. 

479
00:24:47,758 --> 00:24:49,828
So maybe tell us the danger of 
this. 

480
00:24:51,052 --> 00:24:53,092
Totally. 
Let's start with just the 
 

481
00:24:53,100 --> 00:24:56,726
definition of a proxy metric. 
You want a piece of information,

482
00:24:56,726 --> 00:24:59,062

 but it's really hard to get 
at. 

483
00:24:59,102 --> 00:25:02,940
So for example, quality. 
There is no one measure that 
 

484
00:25:02,948 --> 00:25:05,320
tells you about quality. 
Quality is a multidimensional 

485
00:25:05,320 --> 00:25:08,019
thing. 
But if you're looking for a way 

486
00:25:08,019 --> 00:25:11,682
to 
 measure quality, you need a
proxy, something that will kind 

487
00:25:11,682 --> 00:25:14,512
of approximate 
 some 
information about that thing. 

488
00:25:15,152 --> 00:25:18,672
So there's no way to get away 
 
from proxy metrics completely. 

489
00:25:18,672 --> 00:25:20,652
We use proxy metrics all the 
time. 

490
00:25:20,732 --> 00:25:25,242
But when you don't realize 
 
that it's a proxy, that's when 

491
00:25:25,242 --> 00:25:28,022
you end up with problems. 
So let's come back to velocity. 

492
00:25:28,442 --> 00:25:31,404
So the way velocity is usually 
counted 
 in most organizations 

493
00:25:31,404 --> 00:25:34,748
that I see anyway is they've got
some method of 
 pointing, 

494
00:25:34,748 --> 00:25:37,671
whether it's the Fibonacci 
series or whatever, they've got 

495
00:25:37,671 --> 00:25:42,480
some 
 way of saying, this story
is a three, this story is an 

496
00:25:42,480 --> 00:25:46,372
eight, whatever. 
And then they take the sto- the 

497
00:25:46,372 --> 00:25:50,521
 numbers associated with stories
and sum that up, sum up all of 

498
00:25:50,521 --> 00:25:53,361
the stories 
 delivered in a 
given sprint, time period, 

499
00:25:53,361 --> 00:25:56,450
whatever, and say, "That is how 
 fast we can go." 

500
00:25:56,450 --> 00:25:59,982
All of this started, again, with
the best of intentions. 

501
00:26:00,032 --> 00:26:06,669
This was way better than the 
very 
 heavyweight, massive work

502
00:26:06,669 --> 00:26:10,139
breakdown structure method of 
doing project planning 
 that 

503
00:26:10,139 --> 00:26:17,237
was common in the late 1990s. 
But the problem is that, the 

504
00:26:17,237 --> 00:26:20,522
velocity 
 numbers, they're 
made-up numbers. 

505
00:26:20,542 --> 00:26:24,231
They are our best attempt at 
sizing 
 a thing at the moment 

506
00:26:24,231 --> 00:26:28,254
that we know the least amount of
the thing, 
 least amount about 

507
00:26:28,254 --> 00:26:31,269
the thing. 
So then we take these made-up 

508
00:26:31,269 --> 00:26:33,152
numbers 
 and we perform math 
upon them. 

509
00:26:34,242 --> 00:26:36,532
They look very scientific at 
this point. 

510
00:26:36,662 --> 00:26:39,504
They are not. 
When you take made-up numbers 

511
00:26:39,504 --> 00:26:42,946
and you 
 perform math upon 
made-up numbers, you still have 

512
00:26:42,946 --> 00:26:45,181
made-up numbers. 
But if you put them on a chart… 

513
00:26:46,316 --> 00:26:55,372
They look so official, right? 
And that's why I'm not a fan of 

514
00:26:55,372 --> 00:26:58,188
velocity 
 at all because it's 
so easy to game. 

515
00:26:58,198 --> 00:27:00,722
If somebody puts pressure on you
have 
 to increase your 

516
00:27:00,722 --> 00:27:03,538
velocity, I know exactly how to 
increase my velocity. 

517
00:27:03,918 --> 00:27:06,744
That three-point story, nope, 
that 
 was, that's actually an 

518
00:27:06,744 --> 00:27:10,612
eight. 
And without… And now it actually

519
00:27:10,612 --> 00:27:13,812
takes 
 longer to get anything 
done because we're arguing more 

520
00:27:13,812 --> 00:27:16,706
about what the velocity 
 
numbers, like what the points 

521
00:27:16,706 --> 00:27:20,491
should be. 
And so I don't see any 
 good 

522
00:27:20,491 --> 00:27:23,642
coming from this. 
So I much prefer cycle time 

523
00:27:23,642 --> 00:27:26,258
because 
 now you actually have 
a timestamp. 

524
00:27:26,838 --> 00:27:31,428
This is still a proxy for 
productivity. 

525
00:27:31,698 --> 00:27:35,514
Like by itself cycle time does 
not 
 actually tell you if we're

526
00:27:35,514 --> 00:27:40,508
using our time well, but it 
gives you a much better 
 sense 

527
00:27:40,508 --> 00:27:45,038
for from the time that we 
started working on this story to

528
00:27:45,038 --> 00:27:47,998
the time that 
 we delivered it,
what is that time? 

529
00:27:48,108 --> 00:27:53,482
And you may discover that 
between lead time, 
 from the 

530
00:27:53,482 --> 00:27:57,070
time we asked for the story to 
the time it's in production, and

531
00:27:57,070 --> 00:28:00,290
cycle 
 time, from the time we 
started working to the time we 

532
00:28:00,290 --> 00:28:02,266
delivered for whatever 
 
delivered means in your 

533
00:28:02,266 --> 00:28:05,704
organization, that between those
two you may get 
 some really 

534
00:28:05,704 --> 00:28:08,328
interesting information. 
Like maybe the minimum lead 

535
00:28:08,328 --> 00:28:12,838
time, 
 even for the highest 
priority thing, turns out to be 

536
00:28:12,838 --> 00:28:16,284
like 140 days. 
That is interesting information,

537
00:28:16,284 --> 00:28:20,804

 and that is way more 
interesting than our average 

538
00:28:20,804 --> 00:28:24,398
velocity is 37.6. 
Yeah. 

539
00:28:24,858 --> 00:28:27,443
So I think the trend in the 
industry, 
 definitely, the way 

540
00:28:27,443 --> 00:28:30,498
I see it is that software, the 
productivity, development 
 

541
00:28:30,506 --> 00:28:34,908
productivity is something that 
is still kind of like the golden

542
00:28:34,908 --> 00:28:36,585
metrics, right? 
All every engineering 

543
00:28:36,585 --> 00:28:38,918
organizations 
 want to improve 
their productivity. 

544
00:28:39,058 --> 00:28:41,813
And that's why every time 
there's a new 
 research coming 

545
00:28:41,813 --> 00:28:46,070
out on how to quantify, how to 
measure productivity, be it, you

546
00:28:46,070 --> 00:28:49,698

 know, story points, you know, 
velocity, or recently the DORA 

547
00:28:49,698 --> 00:28:53,266
metrics, and maybe 
 lately it's
the number of lines of code or 

548
00:28:53,266 --> 00:28:56,642
tokens, will always be there. 
And people think it's a silver 

549
00:28:56,642 --> 00:28:58,548
bullet 
 that they can use to 
measure, right? 

550
00:28:58,738 --> 00:29:02,075
So how about DORA metrics? 
Because this is also something 

551
00:29:02,075 --> 00:29:05,828
quite, you 
 know, referred by a
lot of organizations. 

552
00:29:05,858 --> 00:29:08,288
Is it something-- Cycle time is 
 definitely one of the DORA 

553
00:29:08,288 --> 00:29:11,320
metric. 
But is it overall a good kind 
 

554
00:29:11,328 --> 00:29:14,712
of like proxy metrics to measure
software development 

555
00:29:14,712 --> 00:29:17,735
productivity? 
I love the DORA metrics, and the

556
00:29:17,735 --> 00:29:21,415

 reason is because they really 
are more about the outcome than 

557
00:29:21,415 --> 00:29:25,353
about outputs, 
 which lines of 
code is an output. 

558
00:29:26,073 --> 00:29:29,483
Token… Don't even get me started

 on tokenmaxxing, 'cause that's

559
00:29:29,483 --> 00:29:31,483
not even, that's not even an 
output. 

560
00:29:32,213 --> 00:29:35,923
That's a sheer, that's a game. 
Yeah. 

561
00:29:36,183 --> 00:29:40,258
Oh, with bad prizes. 
Play stupid games, get stupid 

562
00:29:40,258 --> 00:29:41,513
prizes. 
That's just… No. 

563
00:29:42,873 --> 00:29:46,573
But the DORA metrics 
 are 
focused on outcomes. 

564
00:29:47,233 --> 00:29:50,653
It's not really sufficient to 
steer your 
 whole organization,

565
00:29:50,653 --> 00:29:55,615
but it is way better than a lot 
of the measures that were used 


566
00:29:55,623 --> 00:29:57,403
before "Accelerate" the book 
came out. 

567
00:29:57,443 --> 00:30:01,183
So I'm a huge fan. 
I like the DORA metrics as well.

568
00:30:01,223 --> 00:30:05,316
I think they are kind of more 
 
right side in value stream than 

569
00:30:05,316 --> 00:30:08,343
holistic value stream. 
That's not a… But like Elisabeth

570
00:30:08,343 --> 00:30:10,829
said, 
 they're still much 
better than, you know, kind of 

571
00:30:10,829 --> 00:30:15,239
anything we've had before. 
While I, and I do like them, I 

572
00:30:15,239 --> 00:30:18,008
think 
 it's sometimes it's 
almost even simpler though for 

573
00:30:18,008 --> 00:30:19,953
organizations. 
Instead of like, if they 

574
00:30:19,953 --> 00:30:21,938
actually 
 wanna measure 
developer productivity, instead 

575
00:30:21,938 --> 00:30:24,633
of asking what everybody 
 else 
is doing and then following the 

576
00:30:24,633 --> 00:30:27,529
leader, this is what executive 
and 
 engineering leadership 

577
00:30:27,529 --> 00:30:29,928
should be doing. 
They should be asking, what 
 

578
00:30:29,936 --> 00:30:33,112
does productivity mean to us? 
How would we wanna… Like how 

579
00:30:33,112 --> 00:30:35,263
would we 
 know we are more 
productive than not? 

580
00:30:35,293 --> 00:30:37,648
And then get into the causal 
modeling, the 
 systems thinking

581
00:30:37,648 --> 00:30:41,133
of it, of about, talking about 
what inside of our organization 

582
00:30:41,133 --> 00:30:43,823
is 
 influencing our ability to 
be productive. 

583
00:30:44,223 --> 00:30:47,593
'Cause that, that… if you just 

measure a number, lines of code 

584
00:30:47,603 --> 00:30:51,067
or something, then what do you 
do 
 if the number is not high 

585
00:30:51,067 --> 00:30:53,279
enough? 
You tell them, "Well, write more

586
00:30:53,279 --> 00:30:55,727

 lines of code." 
And that's not helpful, and 

587
00:30:55,727 --> 00:30:58,325
that's not leadership. 
And so like when I think about 

588
00:30:58,325 --> 00:31:00,263
it, it's 
 you actually have to 
study the system. 

589
00:31:00,383 --> 00:31:01,983
Like what does productivity mean
to us? 

590
00:31:02,003 --> 00:31:04,940
What's influencing it? 
And as soon as you talk about 
 

591
00:31:04,948 --> 00:31:07,935
what's influencing it, you see 
this world of levers, this world

592
00:31:07,935 --> 00:31:11,083

 of options available to you. 
It might be things like our 

593
00:31:11,083 --> 00:31:13,573
environments 
 aren't as good as
we thought them to be. 

594
00:31:13,803 --> 00:31:16,323
People can't spin things up. 
There's no isolation. 

595
00:31:16,323 --> 00:31:18,953
The architecture isn't right. 
There's fuzzy requirements. 

596
00:31:18,973 --> 00:31:21,183
You know, there's, the 
 
feedback loops are too big. 

597
00:31:21,193 --> 00:31:25,523
All of a sudden you get these 
things that 
 in your context 

598
00:31:25,523 --> 00:31:29,300
make you more productive. 
And that is so much better than 

599
00:31:29,300 --> 00:31:32,375
 saying, I read Netflix does 
this one thing on Tuesdays, we 

600
00:31:32,375 --> 00:31:35,105
should try that. 
Like, it's just so much more 

601
00:31:35,105 --> 00:31:37,183
better 
 if you understand what 
you're doing. 

602
00:31:37,881 --> 00:31:41,026
Well, and my favorite part about
that, 
 Joel, is I'm thinking 

603
00:31:41,026 --> 00:31:44,631
now about the number of times 
that I've been either 
 in an 

604
00:31:44,631 --> 00:31:47,605
organization or consulting for 
an organization where they 

605
00:31:47,605 --> 00:31:51,191
actually 
 knew the answer to 
the question about what's 

606
00:31:51,191 --> 00:31:52,565
getting in their way. 
Yeah. 

607
00:31:53,051 --> 00:31:55,641
But they viewed that as too 
 
hard a problem to solve. 

608
00:31:55,671 --> 00:31:57,791
Right. 
And so they didn't. 

609
00:31:57,881 --> 00:32:00,489
So like you mentioned 
environments, 
 like, "Oh, don't

610
00:32:00,489 --> 00:32:04,129
talk to me about environments 
anymore because we can't 
 

611
00:32:04,137 --> 00:32:08,475
afford…" This was, I'm thinking 
about an organization way before

612
00:32:08,475 --> 00:32:12,611
containers 
 were a thing and 
spinning up containers and VMs 

613
00:32:12,611 --> 00:32:17,384
and whatever was super easy. 
So to be fair, it cost a lot of 

614
00:32:17,384 --> 00:32:20,124
 money to stand up new 
environments because it meant 

615
00:32:20,124 --> 00:32:23,771
you had to get a new 
 rack in 
the data center and, right? 

616
00:32:24,861 --> 00:32:26,451
So don't talk to me about 
environments. 

617
00:32:26,451 --> 00:32:29,322
We have to solve the problem 
 
without adding more 

618
00:32:29,322 --> 00:32:34,406
environments. 
Okay, but that constraint is 

619
00:32:34,406 --> 00:32:40,622
actually 
 the constraint in 
your organization, and anything 

620
00:32:40,622 --> 00:32:45,531
else you do that doesn't 
 
address that constraint is just 

621
00:32:45,531 --> 00:32:48,461
gonna make that constraint hurt 
even more. 

622
00:32:50,830 --> 00:32:52,960
Right. 
Frankly speaking, I mean, 

623
00:32:52,960 --> 00:32:55,830
borrowing 
 from my experience 
as well, so I was also kind of 

624
00:32:55,830 --> 00:32:57,570
like guilty 
 on this kind of 
thing, right? 

625
00:32:57,570 --> 00:33:01,446
Because things that are quite 
difficult 
 to kind of like 

626
00:33:01,446 --> 00:33:04,510
first observe, right, identify 
what's the issue. 

627
00:33:04,800 --> 00:33:07,270
You will need to deal with a lot
of 
 people, not to mention 

628
00:33:07,270 --> 00:33:11,000
maybe there are also executives 
who may have power, politics, 

629
00:33:11,000 --> 00:33:13,889
and all that. 
And just to come up with the, 

630
00:33:13,889 --> 00:33:16,616
you know, 
 the idea of, you 
know, understanding your system,

631
00:33:16,616 --> 00:33:19,262
understanding your signals, 
 
probably is very, very 

632
00:33:19,262 --> 00:33:21,852
difficult. 
And maybe most of us would not 


633
00:33:21,860 --> 00:33:26,268
attempt that, unless it's like a
very clear that we have to do 

634
00:33:26,268 --> 00:33:28,839
something. 
And usually it's the crisis that

635
00:33:28,839 --> 00:33:31,310

 force us, and then become a 
policy. 

636
00:33:31,580 --> 00:33:33,680
So I think in your book 
 you 
have a better way. 

637
00:33:33,950 --> 00:33:37,840
You have this CREATE framework. 
Tell us about this so that we 
 

638
00:33:37,848 --> 00:33:40,370
can use it rather than, you 
know, fighting against each 

639
00:33:40,370 --> 00:33:43,149
other. 
The CREATE framework's 
 really 

640
00:33:43,149 --> 00:33:45,273
interesting, Henry. 
So I'll give you the, quick 

641
00:33:45,273 --> 00:33:47,181
background 
 story, which 
Elisabeth always loves, and then

642
00:33:47,181 --> 00:33:49,759
we can get into how to use it. 
When Elisabeth first started 

643
00:33:49,759 --> 00:33:52,872
talking 
 about this and she was
just… We, She was trying to kind

644
00:33:52,872 --> 00:33:55,175
of form it. 
I'm not gonna lie to you, it 
 

645
00:33:55,183 --> 00:33:57,915
looked like a radar chart to me.
And so when I first saw it, I 

646
00:33:57,915 --> 00:34:00,123
was 
 like, "Elisabeth," I was 
like, "I can't stand radar 

647
00:34:00,123 --> 00:34:02,127
charts." 
 
I'm like, "They are overused. 

648
00:34:02,567 --> 00:34:05,825
They are, they're not helpful." 
And 
 then as I started getting 

649
00:34:05,825 --> 00:34:10,513
into it, I started realizing how
powerful 
 this CREATE framework

650
00:34:10,513 --> 00:34:13,603
could be. 
And for the readers who haven't 

651
00:34:13,603 --> 00:34:16,197
 seen the book, I don't think it
even looks that way anymore. 

652
00:34:16,237 --> 00:34:18,822
An early sketch of the CREATE 
framework 
 actually looks like 

653
00:34:18,822 --> 00:34:21,692
a radar chart. 
But with the CREATE framework, 

654
00:34:21,692 --> 00:34:25,327
it was 
 meant to be was just 
different dimensions to be 

655
00:34:25,327 --> 00:34:28,069
considering when you're looking 
 at making kind of decisions, 

656
00:34:28,069 --> 00:34:31,782
with the idea being that you 
really don't wanna 
 have, call 

657
00:34:31,782 --> 00:34:34,342
it unintended consequences. 
So the CREATE framework, keep 
 

658
00:34:34,350 --> 00:34:38,043
me honest here Elisabeth, it's 
Capacity, Risk, Execution, 
 

659
00:34:38,052 --> 00:34:40,956
Adaptability, Trust, and 
Economics. 

660
00:34:41,275 --> 00:34:43,206
Yes. 
And so to give a very 
 clear 

661
00:34:43,206 --> 00:34:46,521
example of that. 
Sometimes a very common pattern,

662
00:34:46,521 --> 00:34:48,956

 engineering is going too slow,
right? 

663
00:34:49,507 --> 00:34:51,777
Engineer's going too slow, 
 we 
need to ship sooner. 

664
00:34:52,177 --> 00:34:56,422
And so some people confuse a 
capacity 
 problem -- we don't 

665
00:34:56,422 --> 00:34:59,075
have enough people, we don't 
have enough resources, we 
 

666
00:34:59,083 --> 00:35:02,541
don't have enough systems -- 
sometimes people confuse that 

667
00:35:02,541 --> 00:35:05,227
problem with 
 an execution 
problem, right? 

668
00:35:05,547 --> 00:35:08,713
So sometimes what happens is 
when 
 you think the problem is 

669
00:35:08,713 --> 00:35:11,127
you need more people, you bring 
in more teams. 

670
00:35:11,417 --> 00:35:14,299
But if it's not a capacity 
 
problem, it's an execution 

671
00:35:14,299 --> 00:35:17,295
problem, bringing in more people
makes all 
 the execution 

672
00:35:17,295 --> 00:35:19,626
problems harder. 
Now you have more merge 

673
00:35:19,626 --> 00:35:21,222
conflicts. 
Now you have more environment 

674
00:35:21,222 --> 00:35:22,397
conflicts. 
Testing is harder. 

675
00:35:22,407 --> 00:35:24,907
The data's messed up to test. 
Shipping's harder. 

676
00:35:25,297 --> 00:35:29,157
And so if you think about the C 
in 
 CREATE was Capacity, and 

677
00:35:29,157 --> 00:35:33,377
one of the E is Execution, we 
brought this out there so 
 

678
00:35:33,385 --> 00:35:36,435
people could consider multi 
dimensions of the decisions they

679
00:35:36,435 --> 00:35:39,197
were making so 
 they didn't end
up with being blindsided. 

680
00:35:39,677 --> 00:35:42,787
Same thing with like risk and 
adaptability 
 or, you know, 

681
00:35:42,787 --> 00:35:44,934
trust and economics. 
There's all these kind of 

682
00:35:44,934 --> 00:35:46,445
facets. 
It's not to say that they're 

683
00:35:46,445 --> 00:35:49,093
just two. 
In many decisions, there's multi

684
00:35:49,093 --> 00:35:52,797

 dimensions to be considered. 
And so that's how we came up 

685
00:35:52,797 --> 00:35:54,967
with 
 framework and kinda 
layered it in. 

686
00:35:55,995 --> 00:35:58,333
Yeah. 
My goal as… Part of where it 
 

687
00:35:58,341 --> 00:36:02,741
came from in my own head was 
just trying to be, to figure out

688
00:36:02,741 --> 00:36:04,545
how to 
 characterize different 
contexts. 

689
00:36:05,095 --> 00:36:08,365
Because it- every context is 
different. 

690
00:36:08,665 --> 00:36:13,084
So when you think about some 
 
extremes, like you've got a 

691
00:36:13,084 --> 00:36:16,175
software as a service company 
where they're 
 relatively 

692
00:36:16,175 --> 00:36:20,664
young, they don't have a huge 
number of customers in 

693
00:36:20,664 --> 00:36:22,461
production. 
Maybe they aren't doing 
 

694
00:36:22,469 --> 00:36:24,900
anything that's regulated. 
They can move fast and break 

695
00:36:24,900 --> 00:36:27,109
things. 
And then by contrast, you have 

696
00:36:27,109 --> 00:36:30,271
somebody 
 who is in the 
financial space, been there for 

697
00:36:30,271 --> 00:36:32,725
forever, and people are 
 
trusting them with their money. 

698
00:36:33,315 --> 00:36:36,410
Very different contexts. 
And how do you really… You can 


699
00:36:36,418 --> 00:36:38,705
say it's a different context, 
and that's kind of obvious. 

700
00:36:38,765 --> 00:36:43,195
But when you think about how to 
 characterize what's different, 

701
00:36:44,155 --> 00:36:47,895
then it becomes kind of like 
this 
 meandering, "Well, you 

702
00:36:47,895 --> 00:36:52,031
know, you have to bear in mind 
regulated and…" 
 Okay, let's 

703
00:36:52,031 --> 00:36:56,085
find another way to talk about 
the dimensions of context. 

704
00:36:56,125 --> 00:37:00,372
And that, that was where I was 
in 
 my head when I drew a thing

705
00:37:00,372 --> 00:37:03,048
that looked like a radar chart, 
and Joel 
 had an allergic 

706
00:37:03,048 --> 00:37:07,133
reaction to it. 
But then as we were talking 

707
00:37:07,133 --> 00:37:11,371
through 
 this, that's when we 
started realizing that one of 

708
00:37:11,371 --> 00:37:15,207
the biggest sources of 
 
unintended consequences occurs 

709
00:37:15,207 --> 00:37:19,379
when we think we have a problem 
in one 
 dimension and address 

710
00:37:19,379 --> 00:37:22,241
the problem in that dimension, 
but the problem was actually 
 

711
00:37:22,249 --> 00:37:23,695
in a completely different 
dimension. 

712
00:37:23,735 --> 00:37:27,015
And Risk and Adaptability 
 are 
a good example also. 

713
00:37:27,635 --> 00:37:29,955
Joel mentioned Capacity and 
Execution. 

714
00:37:30,495 --> 00:37:35,259
But the things that we do to 
reduce 
 Risk may actually be 

715
00:37:35,259 --> 00:37:39,735
inhibiting our Adaptability so 
much that 
 they create more 

716
00:37:39,735 --> 00:37:42,385
problems, and potentially even 
increase Risk. 

717
00:37:42,485 --> 00:37:45,458
Yeah. 
I mean, if you think about all 

718
00:37:45,458 --> 00:37:47,935
the 
 change control boards 
that, we put a change control 

719
00:37:47,935 --> 00:37:50,746
board in place to 
 reduce risk,
and therefore we can- we don't 

720
00:37:50,746 --> 00:37:53,581
ship as often because the work 

has to wait for the board to 

721
00:37:53,581 --> 00:37:56,317
come up. 
But now you have work that's 
 

722
00:37:56,325 --> 00:37:59,557
ready to be delivered that you 
might not know if it's meeting 

723
00:37:59,557 --> 00:38:01,167
the 
 customer's needs for an 
extra month. 

724
00:38:01,197 --> 00:38:03,317
And so you don't get 
 feedback 
for another month. 

725
00:38:03,317 --> 00:38:06,489
And now you've, to- by trying to

 control risk, you've lost 

726
00:38:06,489 --> 00:38:08,679
adaptability, and now you have 
higher market risk 
 because 

727
00:38:08,679 --> 00:38:10,527
your customers aren't-- You're 
not getting feedback fast 

728
00:38:10,527 --> 00:38:12,327
enough, and 
 your customers 
might not be happy. 

729
00:38:12,567 --> 00:38:15,467
So again, it's just a, it's a 
 
beautiful kind of simple way. 

730
00:38:16,017 --> 00:38:19,147
By no means are we saying it is 
the 
 end-all be-all, but I 

731
00:38:19,147 --> 00:38:22,679
think we're offering it up as a 
nice way of looking 
 at these 

732
00:38:22,679 --> 00:38:24,837
dimensions of context and 
avoiding unintended 

733
00:38:24,837 --> 00:38:27,068
consequences. 
Right. 

734
00:38:27,468 --> 00:38:30,978
So definitely quite kind of like
quite 
 interesting, right, the 

735
00:38:30,978 --> 00:38:33,257
different kind of like 
variables, within 
 the context 

736
00:38:33,257 --> 00:38:36,804
that you can look at. 
So I think for some it's kind 
 

737
00:38:36,812 --> 00:38:40,360
of like intuitive, right? 
Like fo- so for example, 

738
00:38:40,360 --> 00:38:42,288
capacity, 
 execution, 
economics, right? 

739
00:38:42,708 --> 00:38:45,788
Maybe it's kind of like easy to 
 quantify, but how about 

740
00:38:45,788 --> 00:38:49,138
quantifying risk, adaptability, 
and trust, right? 

741
00:38:49,428 --> 00:38:51,118
So like you mentioned about 
radar chart. 

742
00:38:51,118 --> 00:38:54,416
I assume there's some metrics, 
some 
 number that represent, 

743
00:38:54,416 --> 00:38:56,848
you know, whether we are doing 
good or not. 

744
00:38:57,048 --> 00:38:58,568
So how do you quantify some of 
these? 

745
00:38:58,568 --> 00:39:01,992
Do you have like a question 
 
with survey or, you know, how do

746
00:39:01,992 --> 00:39:07,383
we use this framework? 
So for adaptability, I, my… So 

747
00:39:07,383 --> 00:39:11,597
let 
 me back up and say it 
isn't a radar chart, and we 

748
00:39:11,597 --> 00:39:14,079
don't have metrics 
 for each 
one of these dimensions. 

749
00:39:14,089 --> 00:39:16,309
So I just want it truth in 
advertising. 

750
00:39:16,409 --> 00:39:19,847
I don't want anybody to think 
that 
 this book is gonna tell 

751
00:39:19,847 --> 00:39:21,709
them exactly how to measure all 
these dimensions. 

752
00:39:22,909 --> 00:39:26,125
The framework really gives you a
little 
 bit more of a… the 

753
00:39:26,125 --> 00:39:29,516
intention is take a holistic 
view and think through the 
 

754
00:39:29,524 --> 00:39:33,236
potential implications of when 
you make a change over here, 

755
00:39:33,236 --> 00:39:37,021
what are you putting…. 
 
well, to overuse the word risk, 

756
00:39:37,021 --> 00:39:38,399
what are you putting at risk 
over there? 

757
00:39:38,939 --> 00:39:42,296
But coming to Risk and 
Adaptability, 
 let's start with

758
00:39:42,296 --> 00:39:45,801
Adaptability. 
I love what I think of as the 
 

759
00:39:45,809 --> 00:39:50,769
tracer bullet test, which is, 
how long would it take to make 

760
00:39:50,769 --> 00:39:52,729
the 
 simplest non-functional 
change. 

761
00:39:53,119 --> 00:39:57,799
For example, just changing a 
 
prompt on a field kind of thing.

762
00:39:57,809 --> 00:40:00,379
It's the… There's no behavior 
change. 

763
00:40:00,419 --> 00:40:02,199
We just have to change some 
text. 

764
00:40:02,629 --> 00:40:07,119
How long does it take for that 
to get all 
 the way through the

765
00:40:07,119 --> 00:40:09,971
delivery system from the time 
the ticket is created to the 

766
00:40:09,971 --> 00:40:11,859
time 
 that we see that change 
in production? 

767
00:40:12,249 --> 00:40:15,243
That's gonna give you a pretty 
good sense 
 of your 

768
00:40:15,243 --> 00:40:19,261
adaptability as an organization.
If that is measured in minutes, 

769
00:40:19,261 --> 00:40:23,638
and 
 in some organizations it 
could be potentially measured in

770
00:40:23,638 --> 00:40:27,579
minutes, you're 
 gonna be a 
highly adaptable organization. 

771
00:40:27,589 --> 00:40:32,859
You may have other issues, but 

being able to ship a change is 

772
00:40:32,859 --> 00:40:37,929
not your constraint here. 
By contrast, I worked with an 

773
00:40:37,929 --> 00:40:40,589
organization 
 that in general 
was very agile. 

774
00:40:41,759 --> 00:40:47,029
In general, very, very, good 
 
at turning on a dime as needed. 

775
00:40:47,419 --> 00:40:52,013
But that tracer bullet test to 
go from 
 the time we identified

776
00:40:52,013 --> 00:40:56,253
a change to the time we could 
see it in production 
 for the 

777
00:40:56,253 --> 00:41:00,069
simplest possible change was 
still well over two weeks, which

778
00:41:00,069 --> 00:41:03,431

 was a surprise to that 
organization because their 

779
00:41:03,431 --> 00:41:07,304
self-identity was, "We 
 are 
super adaptive." 

780
00:41:07,304 --> 00:41:11,781
Well, some stuff has happened 
and things aren't as 
 adaptive 

781
00:41:11,781 --> 00:41:16,502
as you think they are anymore. 
I love that example. 

782
00:41:16,542 --> 00:41:19,636
Some of like the signals we talk
about, 
 like in the Adaptive 

783
00:41:19,636 --> 00:41:22,594
space or the Trust space, 
sometimes they get into a little

784
00:41:22,594 --> 00:41:24,244

 bit more of the qualitative 
aspect. 

785
00:41:24,314 --> 00:41:27,266
You know, like the… If you think
about, 
 especially on the Trust

786
00:41:27,266 --> 00:41:31,544
side, like the things that, the 
things people say versus 
 the 

787
00:41:31,544 --> 00:41:33,394
actions that actually happen, 
right? 

788
00:41:33,774 --> 00:41:37,424
Pr every- all… When the 
leadership 
 says every team is 

789
00:41:37,424 --> 00:41:40,454
autonomous, but then you see 
that no team can make 
 any 

790
00:41:40,454 --> 00:41:42,913
decision on their own, right? 
Like they have to defer the 

791
00:41:42,913 --> 00:41:44,764
decision. 
Then you feel like there's a 
 

792
00:41:44,772 --> 00:41:47,114
disconnect in the trust space. 
There's something not quite 

793
00:41:47,114 --> 00:41:49,569
right there. 
In the adaptability space, I 

794
00:41:49,569 --> 00:41:51,854
love 
 the example of the tracer
bullets. 

795
00:41:51,864 --> 00:41:55,420
I also like the idea… Like, 
 
for me, adaptability comes a lot

796
00:41:55,420 --> 00:41:57,170
into architecture. 
And so, like, there's 
 

797
00:41:57,178 --> 00:41:59,654
definitely, like, signals around
architecture that you could see.

798
00:42:00,024 --> 00:42:04,284
Can teams independently test 
 
or does d- do- are all of your 

799
00:42:04,284 --> 00:42:07,352
tests have to be regression? 
Because if all of your tests are

800
00:42:07,352 --> 00:42:09,998

 regression, I can't guarantee 
you that your architecture is 

801
00:42:09,998 --> 00:42:12,774
tightly coupled, but 
 chances 
are you're gonna be pretty low 

802
00:42:12,774 --> 00:42:15,622
on that adaptability because you
don't 
 have that independent 

803
00:42:15,622 --> 00:42:18,066
testability aspect. 
And so when we think about 

804
00:42:18,066 --> 00:42:19,544
adaptability, 
 we look for 
different signals. 

805
00:42:19,754 --> 00:42:22,418
In the book, we go through some 
kind of 
 like examples of 

806
00:42:22,418 --> 00:42:25,400
signals in these spaces. 
But again, they're gonna be, 

807
00:42:25,400 --> 00:42:27,038
some are 
 quantitative, some 
are qualitative. 

808
00:42:29,289 --> 00:42:31,749
Right. 
And, Joel, what you're pointing 

809
00:42:31,749 --> 00:42:35,750
out 
 is that that whole signals
and the CREATE framework thing, 

810
00:42:35,750 --> 00:42:39,823
it's less 
 about identifying 
the perfect metrics and more 

811
00:42:39,823 --> 00:42:43,393
about, for your environment, 
 
getting a read on the signals 

812
00:42:43,393 --> 00:42:46,269
that would tell you about that 
dimension. 

813
00:42:46,599 --> 00:42:50,191
And Risk is interesting because 
it 
 encompasses both the what 

814
00:42:50,191 --> 00:42:56,081
are the risks that we have, 
where you might be 
 looking at 

815
00:42:56,081 --> 00:43:00,264
your number of incidents, the 
number of rollback, looking 

816
00:43:00,264 --> 00:43:05,423
again 
 at DORA, the number of 
rollbacks that you've had to do,

817
00:43:05,423 --> 00:43:09,319
failed changes. 
Those are quantitative. 

818
00:43:10,219 --> 00:43:12,575
But you also may have 
qualitative 
 risks that are 

819
00:43:12,575 --> 00:43:17,931
more about what is the market 
sensitivity to a risk 
 here for

820
00:43:17,931 --> 00:43:22,929
the world that we live in. 
And that's where a financial 
 

821
00:43:22,937 --> 00:43:27,614
services company is gonna have a
very different risk profile than

822
00:43:27,614 --> 00:43:33,495
a 
 startup that matches dogs 
for dog play dates or something,

823
00:43:33,495 --> 00:43:35,815
right? 
Like the risk profile 
 is just 

824
00:43:35,815 --> 00:43:38,088
so different. 
So really what you're looking 
 

825
00:43:38,096 --> 00:43:42,453
for is what are the signals that
tell us about this dimension. 

826
00:43:43,858 --> 00:43:45,853
Yeah. 
So speaking about the tracer 
 

827
00:43:45,861 --> 00:43:48,428
bullet example you mentioned, I 
think in your book you refer 
 

828
00:43:48,436 --> 00:43:51,698
that as the done gap, right? 
So the gap between, you know, 

829
00:43:51,698 --> 00:43:55,234
after you 
 finish your work, 
but something to put it into 

830
00:43:55,234 --> 00:43:58,001
production, right, might take, 

you know, processes or maybe 

831
00:43:58,001 --> 00:43:59,788
approval, review, whatever that 
is, right? 

832
00:43:59,788 --> 00:44:02,415
I think it's quite interesting 
for 
 leaders to actually 

833
00:44:02,415 --> 00:44:06,190
identify the gap, this done gap,
right, so that we can 
 actually

834
00:44:06,190 --> 00:44:08,928
improve the systems, how we 
deliver software to production. 

835
00:44:09,478 --> 00:44:13,517
So I also want you to mention 
about 
 these three illusions 

836
00:44:13,517 --> 00:44:17,286
that you mentioned so that all 
of us actually get 
 

837
00:44:17,294 --> 00:44:20,506
enlightened, because sometimes 
we are under curse living in 

838
00:44:20,506 --> 00:44:23,022
this illusion. 
So tell us about these three 
 

839
00:44:23,030 --> 00:44:26,162
illusions so that we always can 
refer to it whenever we make 

840
00:44:26,162 --> 00:44:29,373
decisions. 
The three illusions: illusions 

841
00:44:29,373 --> 00:44:33,061
of 
 progress, illusions of 
predictability, and illusions of

842
00:44:33,061 --> 00:44:36,689
control. 
I'm not even sure how we… I 
 

843
00:44:36,697 --> 00:44:39,751
think it started off as just us 
sharing stories, and then we 

844
00:44:39,751 --> 00:44:41,473
started 
 seeing patterns in the
stories. 

845
00:44:41,473 --> 00:44:44,508
And then at first, Henry, the, 
each of 
 those illusions were 

846
00:44:44,508 --> 00:44:48,621
their own chapter, and then it 
felt like a little bit 
 us too 

847
00:44:48,621 --> 00:44:51,523
much glorifying the illusions. 
And Elisabeth did a wonderful 
 

848
00:44:51,531 --> 00:44:54,696
job of consolidating them down. 
Well, let's be fair. 

849
00:44:54,766 --> 00:44:57,202
I'm gonna interrupt you briefly 
to say 
 we got some early 

850
00:44:57,202 --> 00:45:00,206
feedback on the book that I will
be forever grateful for. 

851
00:45:02,186 --> 00:45:04,870
One of our reviewers read the 
first three 
 chapters and said,

852
00:45:04,870 --> 00:45:08,723
"Could you give me a break? 
Because it was so depressing. 
 

853
00:45:08,731 --> 00:45:12,006
Just, I need a win in here 
somewhere." 

854
00:45:12,056 --> 00:45:15,980
Okay, we'll consolidate that 
into the… 
 So now illusions is 

855
00:45:15,980 --> 00:45:18,426
the first chapter. 
So it's one chapter. 

856
00:45:18,616 --> 00:45:19,716
All right. 
Go ahead, Joel. 

857
00:45:19,736 --> 00:45:22,155
Sorry. 
And so in no particular order, 

858
00:45:22,155 --> 00:45:24,875
because 
 sometimes each ones 
are my different favorites at 

859
00:45:24,875 --> 00:45:27,881
that point in time. 
Like when you think about the 

860
00:45:27,881 --> 00:45:30,835
illusion 
 of progress, it's 
that, you know, we think we're 

861
00:45:30,835 --> 00:45:32,885
getting somewhere, and 
 we 
think we're getting there 

862
00:45:32,885 --> 00:45:34,651
quickly, but we're really not 
getting there. 

863
00:45:35,871 --> 00:45:39,345
It's the individual team's 
velocity 
 or cycle time or 

864
00:45:39,345 --> 00:45:40,895
something. 
They're shipping really fast, 

865
00:45:40,895 --> 00:45:42,931
but 
 the product's going very 
slow, right? 

866
00:45:43,161 --> 00:45:45,366
Or we're shipping lots of 
features, 
 but nobody's 

867
00:45:45,366 --> 00:45:47,895
actually using it. 
It's this illusion of progress, 

868
00:45:47,895 --> 00:45:50,011
 this illusion that we're 
getting somewhere better. 

869
00:45:50,651 --> 00:45:53,885
In some examples, in some 
organizations 
 I've been with, 

870
00:45:53,885 --> 00:45:57,851
what's wild, and Elisabeth has a
wonderful story about this as 
 

871
00:45:57,859 --> 00:46:01,501
well, I've seen organizations 
say, "We're gonna take, adopt a 

872
00:46:01,501 --> 00:46:04,111
microservices 
 architecture. 
Therefore, every team can be on 

873
00:46:04,111 --> 00:46:05,861
their own and they can ship 
 
independently." 

874
00:46:05,861 --> 00:46:09,195
But what's interesting is every 
team is going really fast, but 

875
00:46:09,195 --> 00:46:11,181
the 
 product experience goes 
across services. 

876
00:46:11,521 --> 00:46:15,173
And so it doesn't matter how 
fast 
 each team goes until it 

877
00:46:15,173 --> 00:46:17,221
all comes together to deliver an
experience. 

878
00:46:17,221 --> 00:46:20,049
So it's an illusion of progress.
The teams feel fast, but 
 the 

879
00:46:20,049 --> 00:46:22,831
product feels slow. 
So that's one example there. 

880
00:46:23,441 --> 00:46:25,281
There's the illusion of 
predictability. 

881
00:46:26,321 --> 00:46:29,929
So a wonderful story in the book
around, 
 you know, traversing 

882
00:46:29,929 --> 00:46:32,871
the coast of California and how 
long will it take. 

883
00:46:33,721 --> 00:46:36,618
And, so the idea behind the 
illusion 
 of predictability 

884
00:46:36,618 --> 00:46:39,763
isn't to say estimates are bad, 
because at some 
 point in time 

885
00:46:39,763 --> 00:46:43,961
somebody has to be able to say, 
what's gonna happen when. 

886
00:46:44,511 --> 00:46:46,905
But this idea that we can 
predict 
 everything in 

887
00:46:46,905 --> 00:46:49,491
software, like we're building 
systems that usually haven't 
 

888
00:46:49,499 --> 00:46:52,456
been built before on technology 
that was just built yesterday 

889
00:46:52,456 --> 00:46:54,101
with 
 people that are learning 
it today. 

890
00:46:54,131 --> 00:46:57,096
And so this idea that we can 
predict how 
 it's gonna happen 

891
00:46:57,096 --> 00:47:00,783
is just kind of silly. 
The book gets a little deeper 
 

892
00:47:00,791 --> 00:47:02,829
into predictability where we 
actually start talking about 
 

893
00:47:02,837 --> 00:47:04,635
variability, which is a topic I 
love. 

894
00:47:05,105 --> 00:47:08,746
But for an example, I had one 
group I was 
 working with, 

895
00:47:08,746 --> 00:47:11,447
wonderful people, wonderful 
product owner, love them to 

896
00:47:11,447 --> 00:47:13,804
death. 
The product owner was convinced 

897
00:47:13,804 --> 00:47:17,913
 that the team didn't know what 
they were doing because, they, 

898
00:47:17,913 --> 00:47:20,895
you know, 
 they weren't 
shipping predictably. 

899
00:47:21,472 --> 00:47:23,885
Hey, that's trust. 
Blind trust, there you go. 

900
00:47:24,102 --> 00:47:28,022
Sorry, go ahead No, you're fine.
And so what I did is I looked at

901
00:47:28,022 --> 00:47:30,673

 their cycle time, and I looked
at the variability in their 

902
00:47:30,673 --> 00:47:34,455
cycle time. 
And so I go, the cycle time is 

903
00:47:34,455 --> 00:47:37,215
12 
 days plus or minus 12 days,
right? 

904
00:47:37,215 --> 00:47:42,077
And so, like, so you-- so it's, 
the PO 
 is like, "Why aren't 

905
00:47:42,077 --> 00:47:43,745
they predictable? 
Why aren't they predictable? 

906
00:47:43,745 --> 00:47:46,234
They're not doing their job. 
They need to be predictable." 

907
00:47:46,234 --> 00:47:48,215
The problem 
 wasn't that they 
weren't predictable. 

908
00:47:48,215 --> 00:47:50,185
The problem was they had 
 so 
much variability that 

909
00:47:50,185 --> 00:47:51,825
predictability was out the 
window. 

910
00:47:52,145 --> 00:47:55,187
And so the PO-- So again, this 

illusion of predictability, 

911
00:47:55,187 --> 00:47:57,145
right, is what's kind of sitting
inside there. 

912
00:47:57,775 --> 00:48:00,325
And the last one, the 
 illusion
of control, right? 

913
00:48:00,655 --> 00:48:04,263
The idea that, that we-- if we 
have 
 meetings, the problems 

914
00:48:04,263 --> 00:48:06,835
will go away. 
If we have a risk control 
 

915
00:48:06,843 --> 00:48:09,801
board, we won't have defects. 
Like all of these kinds of 
 

916
00:48:09,809 --> 00:48:12,725
ideas, they make you feel good. 
They make you feel like you 
 

917
00:48:12,733 --> 00:48:15,235
have control of something, but 
they're not changing the system.

918
00:48:15,305 --> 00:48:17,785
And so we call that, you 
 know,
the illusion of control. 

919
00:48:18,525 --> 00:48:20,665
Which one's your favorite, 
 
Elisabeth, today? 

920
00:48:21,450 --> 00:48:23,610
Today, well, I wanna expand on 
that. 

921
00:48:23,620 --> 00:48:26,336
You mentioned the story. 
So we got permission from 
 

922
00:48:26,344 --> 00:48:30,360
Michael Wolfe who has the best 
answer on Quora ever given. 

923
00:48:30,710 --> 00:48:34,688
The question was, "Why does, why
is 
 software so difficult to 

924
00:48:34,688 --> 00:48:37,952
predict?" 
And he wrote this essay that is 

925
00:48:37,952 --> 00:48:42,318
just, as 
 far as I'm concerned 
deserves to be highlighted, so 

926
00:48:42,318 --> 00:48:45,717
that's why we reached out 
 to 
get his permission to quote his 

927
00:48:45,717 --> 00:48:47,940
essay. 
Because it is hilarious. 

928
00:48:47,990 --> 00:48:53,072
The whole thing is hilarious. 
It starts with, okay, so 
 we're

929
00:48:53,072 --> 00:48:56,576
in San Francisco. 
My buddy and I have decided that

930
00:48:56,576 --> 00:49:00,175

 it would be fun to hike down 
the coast of California and meet

931
00:49:00,175 --> 00:49:02,210
up 
 with our friends in Newport
Beach. 

932
00:49:03,140 --> 00:49:07,102
And whipping out a map, it's 
this 
 many miles, and if we do 

933
00:49:07,102 --> 00:49:09,920
so many miles a day, we should 
be there by 
 Sunday, so we'll 

934
00:49:09,920 --> 00:49:12,080
call them and let them know to 
expect us for dinner. 

935
00:49:12,440 --> 00:49:19,356
And then the story goes on 
involving 
 angry sea lions and 

936
00:49:19,356 --> 00:49:26,261
wiped out a trail and having to 
go backtrack 
 inland and 

937
00:49:26,261 --> 00:49:29,995
unforgiving terrain and blisters
and illness and everything 
 

938
00:49:30,003 --> 00:49:35,918
that can go wrong will go wrong.
So the essay itself is 
 

939
00:49:35,926 --> 00:49:38,804
absolutely hilarious. 
But the thing that I didn't know

940
00:49:38,804 --> 00:49:42,441
when I 
 first encountered that 
essay is that it is a wonderful 

941
00:49:42,441 --> 00:49:47,138
send-up of a classic essay 
 
from, I think, the 1960s, "How 

942
00:49:47,138 --> 00:49:50,985
Long is the Coast of Britain?" 
by, catch me if 
 I'm wrong, 

943
00:49:50,985 --> 00:49:54,614
Joel, I think it's Mandelbrot. 
The same guy who did fractals 

944
00:49:54,614 --> 00:49:57,664
wrote this 
 wonderful essay 
about how it's impossible to 

945
00:49:57,664 --> 00:50:00,304
measure the coast of Britain 
because 
 of the fractal nature 

946
00:50:00,304 --> 00:50:04,654
of coastlines. 
So that is my favorite illusion,

947
00:50:04,654 --> 00:50:06,470

 the illusion of 
predictability. 

948
00:50:07,527 --> 00:50:12,861
I'll just stop there, yeah. 
Yeah, so I think we all 

949
00:50:12,861 --> 00:50:15,400
sometimes 
 live in this 
illusion, I think most of the 

950
00:50:15,400 --> 00:50:17,776
time I would say. 
Especially again, like we 
 

951
00:50:17,784 --> 00:50:20,536
don't understand the system, we 
don't use the systems thinking. 

952
00:50:20,536 --> 00:50:24,129
So we think just by pulling one 
lever 
 we can actually kinda 

953
00:50:24,129 --> 00:50:27,111
like predict, make good 
progress, and kind of control, 

954
00:50:27,111 --> 00:50:29,601
right? 
So I think I wanna take the last

955
00:50:29,601 --> 00:50:31,666

 part of our conversation 
speaking about AI definitely, 

956
00:50:31,666 --> 00:50:34,236
right? 
So, because AI is like one big 


957
00:50:34,244 --> 00:50:38,768
variable in the systems now. 
And most of the companies in the

958
00:50:38,768 --> 00:50:42,203

 industry will think AI will 
speed up their development or 

959
00:50:42,203 --> 00:50:44,706
even like reduce 
 the number of
developers they need. 

960
00:50:45,546 --> 00:50:47,286
So what are your point of view 
in this? 

961
00:50:47,306 --> 00:50:49,816
Is it also an illusion of 
 
progress or illusion of 

962
00:50:49,826 --> 00:50:54,099
something that we are living in?
I think it's very real. 

963
00:50:54,189 --> 00:50:57,967
I mean, Claude and I are buds. 
We work together on stuff all 

964
00:50:57,967 --> 00:51:00,653
the time. 
I think that there is a set of 


965
00:51:00,661 --> 00:51:03,669
skills that we are all figuring 
out on a day-to-day basis. 

966
00:51:04,169 --> 00:51:09,119
As the capability of the tools 
change, 
 like practically day 

967
00:51:09,119 --> 00:51:13,356
to day, and the tricks that 
were, the things that people 
 

968
00:51:13,364 --> 00:51:16,589
were learning six months ago, 
like this is how you actually 

969
00:51:16,589 --> 00:51:20,079
make it do what you 
 want it to
do or this is its limitations, 

970
00:51:20,109 --> 00:51:22,919
those things aren't true 
anymore. 

971
00:51:23,299 --> 00:51:27,396
So for example, six months ago, 
the 
 wisdom was that if you 

972
00:51:27,396 --> 00:51:30,652
said, "Oh, pretend that you 
are…" and then you gave it a 
 

973
00:51:30,660 --> 00:51:32,129
persona, that it would behave 
differently. 

974
00:51:32,149 --> 00:51:34,213
And apparently, I mean, I 
haven't tested 
 this 

975
00:51:34,213 --> 00:51:36,813
extensively, but apparently that
is just not the way that it 

976
00:51:36,813 --> 00:51:41,449
works anymore. 
The… how do you save on tokens? 

977
00:51:41,499 --> 00:51:44,339
And what is a good Skill look 
like? 

978
00:51:44,379 --> 00:51:47,229
And all of these things 
 are 
changing day to day. 

979
00:51:48,059 --> 00:51:53,266
None of that invalidates the 
notion 
 that this is in fact 

980
00:51:53,266 --> 00:51:59,073
transformational, and the 
question isn't is this 
 useful 

981
00:51:59,073 --> 00:52:02,479
or not, it's useful. 
It's already proved itself. 

982
00:52:02,499 --> 00:52:05,899
It's useful. 
The question is, for our 

983
00:52:05,899 --> 00:52:10,496
context, given 
 everything we 
care about, given the world we 

984
00:52:10,496 --> 00:52:14,297
live in and the codebase we're 
dealing 
 with and all of the 

985
00:52:14,297 --> 00:52:17,764
peculiarities that are specific 
to the world that, to what 
 we 

986
00:52:17,764 --> 00:52:22,217
have to deal with, what is the 
best way for us to apply this 

987
00:52:22,217 --> 00:52:23,819
technology? 
I think that is the question. 

988
00:52:24,649 --> 00:52:28,119
Not is this… can this save time?
Yes. 

989
00:52:28,599 --> 00:52:30,659
Can this reduce the number 
 of 
developers that we need? 

990
00:52:31,009 --> 00:52:33,019
Maybe. 
Like I'm not gonna say no. 

991
00:52:33,029 --> 00:52:36,917
Does it change the lines 
 about
what responsibilities should be 

992
00:52:36,917 --> 00:52:39,621
for a given role? 
100%. 

993
00:52:39,621 --> 00:52:43,313
And what's the answer? 
I don't know, and even if I did,

994
00:52:43,313 --> 00:52:44,839
it's 
 gonna be different in two
months. 

995
00:52:44,849 --> 00:52:49,007
So that's why it's so important,
more 
 important today than ever

996
00:52:49,007 --> 00:52:52,281
before, to be able to read the 
signals and figure out 
 what 

997
00:52:52,281 --> 00:52:53,709
levers you have at your 
disposal. 

998
00:52:54,411 --> 00:52:57,060
Yeah. 
I mean, AI is-- If I took a step

999
00:52:57,060 --> 00:53:00,328
back 
 and I-- before we even 
get to the AI aspect of it, if 

1000
00:53:00,328 --> 00:53:02,620
you, if we get into like 
 the 
systems and the cause of 

1001
00:53:02,620 --> 00:53:04,931
modeling and we talk a lot about
loops, right? 

1002
00:53:04,981 --> 00:53:08,851
And so there's an idea 
 of 
reinforcing loops. 

1003
00:53:08,881 --> 00:53:12,401
Now rein- a reinforcing loop 
could 
 be virtuous in the sense

1004
00:53:12,401 --> 00:53:14,445
that things are continuously 
getting better, and 
 get 

1005
00:53:14,445 --> 00:53:17,535
better, and better, and better. 
And they could also be vicious 


1006
00:53:17,543 --> 00:53:20,641
in the sense that they just 
kinda get worse, and worse, and 

1007
00:53:20,641 --> 00:53:23,981
worse. 
And so anything you do in kind 

1008
00:53:23,981 --> 00:53:28,557
of one 
 of those loops, if you 
make a change into one of those 

1009
00:53:28,557 --> 00:53:31,121
loops, sometimes a 
 simple 
nudge can take a vicious loop to

1010
00:53:31,121 --> 00:53:33,872
virtuous or a virtuous loop to 

vicious or kind of balance it 

1011
00:53:33,872 --> 00:53:36,911
out. 
And that's the first step was we

1012
00:53:36,911 --> 00:53:39,971
need to 
 understand kind of 
what's happening inside of our 

1013
00:53:39,971 --> 00:53:42,401
organization, where are those 
loops 
 sitting at, and what's 

1014
00:53:42,401 --> 00:53:45,161
influencing them. 
Because whenever you introduce 


1015
00:53:45,169 --> 00:53:49,293
new technology, whether it be 
the platform team or the cloud 

1016
00:53:49,293 --> 00:53:53,160
or 
 AI, it's going to have a 
ripple effect on some of those 

1017
00:53:53,160 --> 00:53:55,995
loops. 
And the risk we always run is by

1018
00:53:55,995 --> 00:53:58,777

 introducing a new technology 
that we take a perhaps a 

1019
00:53:58,777 --> 00:54:01,985
virtuous loop and 
 turn it into
a vicious loop, or a balanced 

1020
00:54:01,985 --> 00:54:05,314
loop and becomes vicious. 
I've been with some 

1021
00:54:05,314 --> 00:54:07,356
organizations 
 where they have 
a great idea. 

1022
00:54:07,896 --> 00:54:10,542
We're gonna have platform teams,

 so therefore the product teams

1023
00:54:10,542 --> 00:54:12,256
can just focus on the product 
stuff. 

1024
00:54:12,806 --> 00:54:15,759
This way the, all the Kubernetes
and 
 cloud stuff is abstracted 

1025
00:54:15,759 --> 00:54:16,936
away, teams don't have to worry 
about it. 

1026
00:54:16,966 --> 00:54:21,540
Wonderful idea until, right, 
until 
 the teams can't deploy 

1027
00:54:21,540 --> 00:54:23,948
because they don't understand 
the errors they're 
 getting, 

1028
00:54:23,948 --> 00:54:27,064
and then they have to wait and 
file a ticket and then wait two 

1029
00:54:27,064 --> 00:54:28,356
 weeks and they can't do 
anything. 

1030
00:54:28,656 --> 00:54:30,856
And so like with the best 
 
intentions, you might introduce 

1031
00:54:30,866 --> 00:54:34,456
something that actually shifts 
your system around. 

1032
00:54:34,786 --> 00:54:37,828
And I think that's where AI is 
gonna 
 end up falling, is that 

1033
00:54:37,828 --> 00:54:40,098
if people-- if leadership 
doesn't understand 
 the system 

1034
00:54:40,098 --> 00:54:43,622
at play, they could introduce AI
in such a way that it 
 

1035
00:54:43,630 --> 00:54:47,384
exacerbates an existing problem.
If you live in existing code 

1036
00:54:47,384 --> 00:54:51,024
base 
 that is already hard to 
reason with and you're getting 

1037
00:54:51,024 --> 00:54:54,222
thousands of 
 lines of code 
jumped at you, and it's already 

1038
00:54:54,222 --> 00:54:57,344
hard to test, probably-- 
 
you're probably gonna have a bad

1039
00:54:57,344 --> 00:55:00,188
time. 
If you're working in a simple 

1040
00:55:00,188 --> 00:55:02,282
system 
 with simple 
architecture and clear domain 

1041
00:55:02,282 --> 00:55:05,306
boundaries and clear context 
 
and clear separation of concerns

1042
00:55:05,306 --> 00:55:08,183
and you have high confidence in 
your 
 testability, and you can 

1043
00:55:08,183 --> 00:55:11,336
work in small steps, AI could be
an amplifier for you. 

1044
00:55:11,406 --> 00:55:13,156
It could help you kind 
 of 
reason through things. 

1045
00:55:13,856 --> 00:55:17,070
There's one interesting thing 
 
that we leave I think maybe as 

1046
00:55:17,070 --> 00:55:19,864
an exercise for the reader. 
There's an interesting question 

1047
00:55:19,864 --> 00:55:23,530
 to think about is, if you can 
ship-- what happens when you 

1048
00:55:23,530 --> 00:55:25,186
ship 
 faster than you learn, 
right? 

1049
00:55:25,216 --> 00:55:27,028
And I think that's a really 
interesting 
 question for us to

1050
00:55:27,028 --> 00:55:29,116
be thinking about in a systems 
kind of world. 

1051
00:55:29,566 --> 00:55:32,092
If you're shipping faster than 
your 
 feedback loop back into 

1052
00:55:32,092 --> 00:55:35,326
your learning cycle of your 
product, what happens? 

1053
00:55:36,826 --> 00:55:38,586
That's something to 
 see and 
kind of follow. 

1054
00:55:40,345 --> 00:55:44,613
Wow, very interesting question. 
What happens if you ship faster 

1055
00:55:44,613 --> 00:55:47,065
than the 
 way you learn from 
the feedback, right? 

1056
00:55:47,665 --> 00:55:50,409
Sometimes even like you ship 
faster, but 
 you don't actually

1057
00:55:50,409 --> 00:55:53,369
remember what you ship. 
I think this Is what I feel as 

1058
00:55:53,369 --> 00:55:54,856
well. 
Yeah. 

1059
00:55:55,015 --> 00:55:57,291
Yeah. 
Yeah. 

1060
00:55:57,611 --> 00:56:01,377
So maybe tell us also, like, 
maybe you 
 have consulted with 

1061
00:56:01,377 --> 00:56:04,541
different people, like seeing in
the industry, how best 
 

1062
00:56:04,549 --> 00:56:07,467
engineering leaders should 
tackle AI, this AI question, 

1063
00:56:07,467 --> 00:56:10,411
right, in, the 
 context of 
systems thinking, right? 

1064
00:56:10,441 --> 00:56:12,281
So I think you mentioned 
 a 
couple of things, right? 

1065
00:56:12,281 --> 00:56:15,191
But maybe tell us the best, 
 
kind of like the best practice. 

1066
00:56:15,191 --> 00:56:17,800
I know it's maybe difficult at 
this 
 moment to come up with 

1067
00:56:17,800 --> 00:56:20,091
the best practice. 
But what have you seen work? 

1068
00:56:20,371 --> 00:56:23,705
Like how do people apply AI in 
the 
 context of systems 

1069
00:56:23,705 --> 00:56:27,225
thinking such that it can create
a reinforcing 
 loop that is 

1070
00:56:27,225 --> 00:56:32,854
like virtuous? 
I have an answer to that, and 

1071
00:56:32,854 --> 00:56:35,782
it's 
 something that I will say
that I have. 

1072
00:56:35,822 --> 00:56:38,262
It has caused me to change 
 my 
mind about something. 

1073
00:56:39,602 --> 00:56:44,262
I think that the way, the best 
way 
 to use AI right now is to 

1074
00:56:44,262 --> 00:56:47,802
think about optimizing for 
learning specifically. 

1075
00:56:48,402 --> 00:56:55,576
And I saw an organization where 
they 
 were committed to AI, 

1076
00:56:55,576 --> 00:56:59,532
using AI all the time, but 
everybody was using it 
 

1077
00:56:59,540 --> 00:57:03,370
differently and they took a week
to do essentially like 

1078
00:57:03,370 --> 00:57:07,660
vibe-athon kind of thing. 
And the whole purpose was to 

1079
00:57:07,660 --> 00:57:11,577
learn. 
There wasn't a, like no prizes, 

1080
00:57:11,577 --> 00:57:13,762
 not the typical hackathon 
thing. 

1081
00:57:14,242 --> 00:57:18,988
But instead, we're gonna take a 
week 
 to learn what happens 

1082
00:57:18,988 --> 00:57:22,237
when we apply AI. 
We're gonna learn from each 

1083
00:57:22,237 --> 00:57:24,276
other. 
We're gonna have a way to come 


1084
00:57:24,284 --> 00:57:25,842
together and share learnings 
every day. 

1085
00:57:26,352 --> 00:57:29,817
And they took a week to do this.
And I realized something, 'cause

1086
00:57:29,817 --> 00:57:34,531
in the 
 past, as an engineering
leader, I was 100% all for 

1087
00:57:34,531 --> 00:57:39,288
taking time to learn, and I 
 
put some pretty clear guardrails

1088
00:57:39,288 --> 00:57:44,424
on it. 
So like we would do a, kind of a

1089
00:57:44,424 --> 00:57:48,362
hack day 
 kind of thing to 
where it's just go play. 

1090
00:57:48,412 --> 00:57:51,584
Go play with technology. 
Go learn like one day a month, 


1091
00:57:51,592 --> 00:57:56,612
or sorry, one day a quarter. 
That is not enough anymore. 

1092
00:57:57,002 --> 00:58:01,531
And so my… now what I believe is

 that you do have to decide for

1093
00:58:01,531 --> 00:58:05,450
your own context what the right 
cadence 
 is and what's the 

1094
00:58:05,450 --> 00:58:09,174
right duration. 
But if you think about we need 


1095
00:58:09,182 --> 00:58:12,136
to optimize for learning, and 
what is it that we need to 

1096
00:58:12,136 --> 00:58:13,409
learn? 
What are the questions 
 that we

1097
00:58:13,409 --> 00:58:15,584
need to answer? 
My personal belief is that at 

1098
00:58:15,584 --> 00:58:20,134
this 
 point, at this moment in 
time, we're in the moment where 

1099
00:58:20,134 --> 00:58:25,158
figuring 
 out harnesses is the 
big question in front of every 

1100
00:58:25,158 --> 00:58:27,049
organization. 
Because ultimately, I believe 
 

1101
00:58:27,057 --> 00:58:30,818
that the general purpose 
frontier models are not gonna be

1102
00:58:30,818 --> 00:58:33,714
how we're 
 developing software 
a year from now. 

1103
00:58:34,434 --> 00:58:37,538
And rather it's gonna be 
harnesses 
 that have many 

1104
00:58:37,538 --> 00:58:40,988
agents that are attached to many
different models, and the 
 

1105
00:58:40,996 --> 00:58:44,168
harness is responsible for 
figuring out what kinds of work 

1106
00:58:44,168 --> 00:58:48,529
goes to which. 
And, in my head, the analogy is 

1107
00:58:48,529 --> 00:58:51,796
like, 
 database query 
optimizers, where it's the 

1108
00:58:51,796 --> 00:58:56,144
database query optimizer 
 that 
knows how to farm out the query.

1109
00:58:56,904 --> 00:58:58,884
Like this is the age 
 that I 
think we're in. 

1110
00:59:00,274 --> 00:59:02,082
We'll see whether or not I'm 
right 
 about that, but that's 

1111
00:59:02,082 --> 00:59:04,294
not the point. 
The point is the learning, 

1112
00:59:04,294 --> 00:59:08,262
figuring out 
 what questions do
we need to answer, and creating 

1113
00:59:08,262 --> 00:59:12,546
more space than you ever 
 
thought was necessary to enable 

1114
00:59:12,546 --> 00:59:16,060
the organization to answer those
questions, 
 I think that is the

1115
00:59:16,060 --> 00:59:19,174
most important thing that any 
organization can be doing today.

1116
00:59:20,577 --> 00:59:24,617
I would add on to that, that I 

would-- So I love the learning 

1117
00:59:24,617 --> 00:59:28,841
aspect for a variety of reasons.
I would off-- also offer up that

1118
00:59:28,841 --> 00:59:31,057
it 
 needs to be done, not 
individually based. 

1119
00:59:31,587 --> 00:59:34,615
It needs to be done at least in 
a 
 pair, if not in a small 

1120
00:59:34,615 --> 00:59:36,905
group base, because otherwise we
kind of get 
 into our own kind 

1121
00:59:36,905 --> 00:59:38,667
of niches of kind of what worked
and what didn't. 

1122
00:59:38,977 --> 00:59:42,046
And we actually do need some 
 
constructive, you know, 

1123
00:59:42,046 --> 00:59:44,057
conflict. 
Conflict-- good conflict, right?

1124
00:59:44,057 --> 00:59:46,447
Like I'm not sure if that's kind

 of saying what we think it is.

1125
00:59:46,467 --> 00:59:50,050
Is that helpful? 
The other thing I would offer up

1126
00:59:50,050 --> 00:59:54,667
for 
 leadership that's looking 
into AI is again, kind of like… 

1127
00:59:54,667 --> 00:59:57,610
AI is a hypothesis 
 for 
something inside of your 

1128
00:59:57,610 --> 00:59:59,447
organization. 
And so whatever you're looking 

1129
00:59:59,447 --> 01:00:03,605
at 
 adopting, you should be 
thinking about how could we find

1130
01:00:03,605 --> 01:00:06,986
out sooner if it's going 
 the 
right way or the wrong way, 

1131
01:00:06,986 --> 01:00:09,123
right? 
Like so how do you find shorter 

1132
01:00:09,123 --> 01:00:12,052
 feedback loops as well as what 
are the countermeasures that you

1133
01:00:12,052 --> 01:00:14,826
want to look 
 out for… Sorry, 
the countermeasures you want to 

1134
01:00:14,826 --> 01:00:17,137
have in place to make 
 sure 
things don't go off the rail. 

1135
01:00:17,137 --> 01:00:20,168
Or what are the other, you know,
if you 
 look at the CREATE 

1136
01:00:20,168 --> 01:00:22,981
framework, if you're bringing an
AI to help with execution, 
 

1137
01:00:22,989 --> 01:00:24,787
does it affect something else, 
right? 

1138
01:00:24,817 --> 01:00:26,972
And so that's the kind of 
systemic 
 things that you want 

1139
01:00:26,972 --> 01:00:30,127
to be thinking about in 
addition, right, is the 
 how do

1140
01:00:30,127 --> 01:00:33,097
I find out sooner if it's going 
where I want it to go? 

1141
01:00:33,117 --> 01:00:35,167
How do I avoid unintended 
consequences? 

1142
01:00:35,167 --> 01:00:37,197
And how do I know we're 
 going 
the right way? 

1143
01:00:38,742 --> 01:00:40,568
Yeah. 
I love optimizing for learning 


1144
01:00:40,576 --> 01:00:42,122
and for the feedback loop, 
right? 

1145
01:00:42,122 --> 01:00:45,172
So definitely something 
 we all
can do, right? 

1146
01:00:45,642 --> 01:00:50,422
And I like the vibeathon idea. 
So I think I haven't heard about

1147
01:00:50,422 --> 01:00:52,312
that 
 before, but I think it 
makes sense now. 

1148
01:00:53,222 --> 01:00:57,789
So, I think one thing that also 
I would 
 like to ask, like 

1149
01:00:57,789 --> 01:01:01,660
system thinking is very hard to 
understand, and sometimes 
 we 

1150
01:01:01,660 --> 01:01:04,840
all, one person, right, it's 
very hard to understand 

1151
01:01:04,840 --> 01:01:07,632
everything, right? 
Can we use AI or have you seen, 

1152
01:01:07,632 --> 01:01:10,814
you know, 
 someone using AI to 
help you in doing systems 

1153
01:01:10,814 --> 01:01:13,644
thinking in their organizations?
And if so, what kind of things 


1154
01:01:13,652 --> 01:01:17,935
that we should feed into the AI?
So I personally have not seen 

1155
01:01:17,935 --> 01:01:20,246
it. 
That's not to say it couldn't 
 

1156
01:01:20,254 --> 01:01:24,434
be done, by all means. 
I'm a big fan of, if we're gonna

1157
01:01:24,434 --> 01:01:27,270
do 
 systems thinking, if we're 
gonna do causal modeling, if 

1158
01:01:27,270 --> 01:01:30,584
we're gonna do 
 a look at 
anything, I'm a big fan of us 

1159
01:01:30,584 --> 01:01:33,614
doing it together because we 
 
need to have different 

1160
01:01:33,614 --> 01:01:34,928
perspectives. 
So like if I'm an engineering 

1161
01:01:34,928 --> 01:01:37,068
leader, 
 I'd wanna do it with a
product leader and maybe, you 

1162
01:01:37,068 --> 01:01:38,983
know, different 
 types of 
leadership so we could talk 

1163
01:01:38,983 --> 01:01:42,848
about the system at large and 
 
talk about how we see the 

1164
01:01:42,848 --> 01:01:45,160
influences and the data we have 
behind them. 

1165
01:01:45,570 --> 01:01:48,330
I'd be nervous. 
I mean, I just don't know. 

1166
01:01:48,710 --> 01:01:52,104
I'd be nervous to see how well 
AI 
 would do that, especially 

1167
01:01:52,104 --> 01:01:55,162
if it was overly confident in 
its results, but 
 I have 

1168
01:01:55,162 --> 01:01:58,931
nothing to back that up on. 
So I have a theory. 

1169
01:02:00,371 --> 01:02:03,821
So first of all, I wanna 
challenge the 
 idea that 

1170
01:02:03,821 --> 01:02:06,671
systems thinking is hard. 
I actually think that every 

1171
01:02:06,671 --> 01:02:08,871
single 
 one of us is a systems 
thinker. 

1172
01:02:08,911 --> 01:02:11,755
I think that is just how we 
navigate 
 the world, because 

1173
01:02:11,755 --> 01:02:15,095
otherwise we would not be able 
to deal with 
 the ambiguity of 

1174
01:02:15,095 --> 01:02:17,761
the universe that we walk 
through every single day. 

1175
01:02:18,831 --> 01:02:21,658
The problem isn't the systems 
 
thinking part, it's 

1176
01:02:21,658 --> 01:02:25,861
articulating… Yes. 
What we're reacting to, being 

1177
01:02:25,861 --> 01:02:29,385
able 
 to explain so that 
somebody else understands the 

1178
01:02:29,385 --> 01:02:33,437
signals that we're seeing. 
And I love, Joel, you've 
 got a

1179
01:02:33,437 --> 01:02:39,755
phrase about, show other people 
what you see… See the same 

1180
01:02:39,755 --> 01:02:41,431
reality. 
See the same reality. 

1181
01:02:42,071 --> 01:02:44,905
Right. 
So making it so that we 
 can 

1182
01:02:44,905 --> 01:02:47,251
see the same reality. 
That is the biggest challenge so

1183
01:02:47,251 --> 01:02:50,023

 that we can navigate the same 
system together, because we all 

1184
01:02:50,023 --> 01:02:51,931
have our 
 own perspectives on 
the system. 

1185
01:02:52,561 --> 01:02:56,099
So when it comes to AI and 
seeing 
 the same reality and 

1186
01:02:56,099 --> 01:02:59,132
being able to articulate what 
we're reacting to, I 
 don't 

1187
01:02:59,132 --> 01:03:03,711
think we can feed it all into AI
and have AI give us the answer. 

1188
01:03:03,711 --> 01:03:06,491
I think that's how we end up 
 
with the answer to the universe 

1189
01:03:06,491 --> 01:03:09,321
is 42, but everybody has 
 
forgotten what the question is. 

1190
01:03:10,821 --> 01:03:15,351
And instead, it can be a 
 
fantastic thinking partner. 

1191
01:03:15,641 --> 01:03:20,527
Like if you say, "Hey, I wanna 
make sure 
 I'm thinking through

1192
01:03:20,527 --> 01:03:24,497
holistically all the signals, 
with respect to capacity, 
 

1193
01:03:24,505 --> 01:03:28,041
risk, execution, adaptability, 
trust, and economics, can you 

1194
01:03:28,041 --> 01:03:32,647
give me a list?" 
 
Give me a list of 20 is one of 

1195
01:03:32,647 --> 01:03:34,931
my favorite things to do with 
ChatGPT. 

1196
01:03:34,961 --> 01:03:38,071
It's amazing at giving me 
 a 
list of 20 whatevers. 

1197
01:03:38,181 --> 01:03:42,301
Like, and-- So I think that the 
way that 
 I would wanna use it 

1198
01:03:42,301 --> 01:03:45,611
is more as a thinking partner 
and less of an answer machine. 

1199
01:03:46,111 --> 01:03:49,411
And as a thinking partner, 
because all 
 of the, especially

1200
01:03:49,411 --> 01:03:52,989
the general purpose frontier 
models have been trained across 

1201
01:03:52,989 --> 01:03:58,216
 such a huge corpus, it's gonna 
be able to surface things that I

1202
01:03:58,216 --> 01:04:01,411
think no one 
 person would've 
been able to surface. 

1203
01:04:01,411 --> 01:04:05,026
And that can help with the 
thinking and 
 articulating 

1204
01:04:05,026 --> 01:04:08,015
amongst a group of people. 
Nice. 

1205
01:04:08,185 --> 01:04:11,749
Love the seeing the reality 
because 
 I find it that's the 

1206
01:04:11,749 --> 01:04:14,945
most difficult thing do in an 
organization. 

1207
01:04:15,695 --> 01:04:18,265
So we have discussed a lot. 
Before we jump into the last 

1208
01:04:18,265 --> 01:04:20,776
question, 
 anything else that 
you wanna call out that we 

1209
01:04:20,776 --> 01:04:23,761
haven't really discussed? 
Anything from the book or maybe 

1210
01:04:23,761 --> 01:04:25,825
some 
 things that you 
encountered recently? 

1211
01:04:26,799 --> 01:04:29,026
Oh boy. 
There's a myriad of things 
 in 

1212
01:04:29,026 --> 01:04:32,588
the top of my head. 
I think one of the things that I

1213
01:04:32,588 --> 01:04:37,574
liked 
 the most from the book, 
something that we, let's say we 

1214
01:04:37,574 --> 01:04:42,128
kinda added late in the book. 
And that is, one of the tools we

1215
01:04:42,128 --> 01:04:44,890
introduce 
 in the book is the 
idea of U-curves, not something 

1216
01:04:44,890 --> 01:04:47,793
that obviously we created. 
It's from economics and we both 

1217
01:04:47,793 --> 01:04:50,946
 are in debt to Don Reinertsen 
for his wonderful explanation of

1218
01:04:50,946 --> 01:04:53,838
it. 
But one thing we did late in the

1219
01:04:53,838 --> 01:04:56,645
book 
 that I thought was really
beautiful is we started talking 

1220
01:04:56,645 --> 01:04:59,704
about showing U-curves 
 and 
showing how context affects 

1221
01:04:59,704 --> 01:05:02,670
U-curves. 
And I think that is a really 
 

1222
01:05:02,678 --> 01:05:03,906
powerful technique for 
leadership. 

1223
01:05:03,966 --> 01:05:07,966
Because like if you wanna ask 
 
the question, how, where should 

1224
01:05:07,966 --> 01:05:10,466
we invest in test automation? 
Well, it depends. 

1225
01:05:10,556 --> 01:05:12,946
What does it depend on? 
Depends on context. 

1226
01:05:13,166 --> 01:05:15,096
How would I know what's the 
right context? 

1227
01:05:15,096 --> 01:05:16,936
And so… Or how do I 
 know 
what's right for me? 

1228
01:05:16,946 --> 01:05:20,246
And so you could use U-curves, 
and you 
 could see in different

1229
01:05:20,246 --> 01:05:23,735
contexts what the appropriate 
blend might be based 
 upon your

1230
01:05:23,735 --> 01:05:26,353
architecture, the coupling of 
the code, your current 

1231
01:05:26,353 --> 01:05:28,066
testability. 
Like, you could make these 

1232
01:05:28,066 --> 01:05:30,363
decisions. 
And what I also love about, 

1233
01:05:30,363 --> 01:05:33,676
kinda 
 how we went through that
is by using U-curves in context,

1234
01:05:33,676 --> 01:05:37,206
you could also see 
 things that
the right decision today could 

1235
01:05:37,206 --> 01:05:39,934
be the wrong decision tomorrow. 
And as Elisabeth very eloquently

1236
01:05:39,934 --> 01:05:43,256

 pointed out to me last week, 
the wrong decision today could 

1237
01:05:43,256 --> 01:05:46,075
actually be the 
 right decision
tomorrow because the context 

1238
01:05:46,075 --> 01:05:49,380
changes and the system adapts. 
And so I think that thing, we 

1239
01:05:49,380 --> 01:05:51,304
haven't 
 really talked about 
it, but this idea of using 

1240
01:05:51,304 --> 01:05:54,249
U-curves in context and starting

 to blend some of these system 

1241
01:05:54,249 --> 01:05:56,927
thinkings tools together, 
there's a world of power 
 

1242
01:05:56,935 --> 01:05:59,666
inside there that I hope our 
readers get. 

1243
01:06:00,979 --> 01:06:02,547
Absolutely. 
The tools are all 

1244
01:06:02,547 --> 01:06:03,799
self-supporting. 
Sorry, go ahead. 

1245
01:06:04,225 --> 01:06:06,395
For people who don't know 
 
U-curves, what is U-curves? 

1246
01:06:06,975 --> 01:06:08,415
'Cause I'm not familiar with 
that as well. 

1247
01:06:09,712 --> 01:06:12,562
Oh, so the… I think the 
 
quickest way I can explain 

1248
01:06:12,572 --> 01:06:16,108
U-curves is when you have 
multiple 
 contributing factors 

1249
01:06:16,108 --> 01:06:19,857
to a decision. 
And so it's seeing those 

1250
01:06:19,857 --> 01:06:22,942
contributing 
 factors create a 
total kinda answer. 

1251
01:06:22,942 --> 01:06:26,132
So if we were to kinda talk 
about, 
 what should the batch 

1252
01:06:26,132 --> 01:06:28,857
size be? 
How big should our batch size 

1253
01:06:28,857 --> 01:06:31,310
be? 
Well, there is contributing 

1254
01:06:31,310 --> 01:06:34,267
factors. 
One contributing factor is 
 the

1255
01:06:34,267 --> 01:06:37,537
cost per unit, right? 
And so if you're doing 

1256
01:06:37,537 --> 01:06:40,648
manufacturing, 
 if you set up 
the machine and you can produce 

1257
01:06:40,648 --> 01:06:44,672
a million screws, the 
 cost per
screw goes down, right? 

1258
01:06:45,162 --> 01:06:48,788
But if you need to, if you have 
a… If 
 you produce a million 

1259
01:06:48,788 --> 01:06:52,162
screws, but you can only sell 
one at a time, you have a 
 

1260
01:06:52,170 --> 01:06:53,982
holding cost of holding all the 
inventory. 

1261
01:06:54,242 --> 01:06:57,535
And so you're… As you increase 
your 
 batch size, your price 

1262
01:06:57,535 --> 01:07:00,292
per unit goes down, but your 
holding cost goes up. 

1263
01:07:00,422 --> 01:07:03,188
And so you see if you add those 
two 
 lines together, you get 

1264
01:07:03,188 --> 01:07:05,517
this U-curve. 
The same thing happens inside 
 

1265
01:07:05,525 --> 01:07:07,999
organizations when we're talking
about how many people should be 

1266
01:07:07,999 --> 01:07:09,802
on a team. 
It's Brooks's law, right? 

1267
01:07:09,802 --> 01:07:12,580
As if you add more late people. 
If you add people to a late 
 

1268
01:07:12,588 --> 01:07:14,752
project, it gets later. 
You can see it because it 
 

1269
01:07:14,760 --> 01:07:16,782
becomes a U-curve problem. 
It gets into testing. 

1270
01:07:16,782 --> 01:07:19,032
It gets into all of 
 these 
wonderful aspects. 

1271
01:07:19,362 --> 01:07:22,908
A U-curve is just multiple 
contributing 
 factors coming up

1272
01:07:22,908 --> 01:07:27,491
to one total value. 
And figuring out the sweet spot,

1273
01:07:27,491 --> 01:07:30,738
it's 
 particularly useful when,
maybe you've got position-based 

1274
01:07:30,738 --> 01:07:34,505
arguments happening in your 
 
organization at two ends of a 

1275
01:07:34,505 --> 01:07:36,681
spectrum. 
Like hypothetically, not that 

1276
01:07:36,681 --> 01:07:40,877
anyone I 
 know has ever had 
this argument, but we should be 

1277
01:07:40,877 --> 01:07:44,037
producing one at a time versus 

we should be producing a 

1278
01:07:44,037 --> 01:07:46,693
gigantic batch. 
Those are two extremes, and the 

1279
01:07:46,693 --> 01:07:50,859
U-curve 
 is a mechanism to find
the sweet spot in between that 

1280
01:07:50,859 --> 01:07:56,897
is specific for your context. 
But a, probably a more realistic

1281
01:07:56,897 --> 01:08:00,761

 debate inside an organization 
would involve something like 

1282
01:08:03,341 --> 01:08:09,051
absolutes related to maybe how 
we 
 think about the release 

1283
01:08:09,051 --> 01:08:11,536
process. 
That we shouldn't think about 

1284
01:08:11,536 --> 01:08:14,599
releasing 
 until we have 
finished everything versus we 

1285
01:08:14,599 --> 01:08:17,871
should be releasing 
 every 
feature as it's completed. 

1286
01:08:17,911 --> 01:08:21,256
And that you can have people who
get 
 really locked into 

1287
01:08:21,256 --> 01:08:23,826
position-based arguments instead
of thinking through, 
 well, 

1288
01:08:23,826 --> 01:08:26,763
actually, there's a whole 
continuum of answers, and the 

1289
01:08:26,763 --> 01:08:29,911
question is where is 
 the right
answer for us on this spectrum. 

1290
01:08:31,078 --> 01:08:34,095
There's that example. 
There's the, every team has to 


1291
01:08:34,104 --> 01:08:38,638
be full stack and has to be able
to do everything versus we have 

1292
01:08:38,638 --> 01:08:41,287
 a team of specialists, right? 
And so like those are 
 two ends

1293
01:08:41,287 --> 01:08:43,596
of the spectrum. 
And the reality is based upon 

1294
01:08:43,596 --> 01:08:46,042
the 
 context of the 
organization as well as even the

1295
01:08:46,042 --> 01:08:48,518
context of the problem, 
 the 
answer is going to be different.

1296
01:08:49,961 --> 01:08:52,337
Yeah. 
So yeah, this is a perfect 

1297
01:08:52,337 --> 01:08:54,809
example 
 of how, you know, 
systems thinking can give us, 

1298
01:08:54,809 --> 01:08:57,350
you know, kind of like 
 an idea
how to kind of like solve 

1299
01:08:57,401 --> 01:08:58,881
problems based on our context, 
right? 

1300
01:08:58,881 --> 01:09:01,430
There's no right or wrong. 
I'm sure there's no silver 
 

1301
01:09:01,439 --> 01:09:04,913
bullet that we can refer to. 
So I think thanks for explaining

1302
01:09:04,913 --> 01:09:06,251
that. 
And thank you for the book. 

1303
01:09:06,251 --> 01:09:09,169
I think for people who love 
systems 
 thinking, this is 

1304
01:09:09,169 --> 01:09:11,020
definitely one of the book you 
should check out. 

1305
01:09:11,751 --> 01:09:15,251
Because, yeah, this is quite 
 
interesting, the way you guys 

1306
01:09:15,251 --> 01:09:20,336
kinda like narrate the concepts.
So as we go to the end of our 
 

1307
01:09:20,344 --> 01:09:23,513
conversation, I have one last 
question, which is like a custom

1308
01:09:23,513 --> 01:09:27,832
in my podcast, 
 which is to ask
you to share with me the three 

1309
01:09:27,832 --> 01:09:30,578
technical leadership wisdom. 
I'll leave it up to you how do 

1310
01:09:30,578 --> 01:09:32,778
you 
 wanna share it, but yeah, 
maybe if you can share, that 

1311
01:09:32,778 --> 01:09:35,617
would be great. 
I'll go first for one, then I'll

1312
01:09:35,617 --> 01:09:38,616
turn 
 it over to Elisabeth and 
we'll see where we… So my first 

1313
01:09:38,616 --> 01:09:42,928
one, Henry, and this 
 is the 
one that I, hopefully Elisabeth 

1314
01:09:42,957 --> 01:09:44,268
isn't tired of hearing me say 
it. 

1315
01:09:46,127 --> 01:09:48,657
I think the takeaway for 
technical 
 leadership is act 

1316
01:09:48,657 --> 01:09:51,127
where the pain is caused, not 
where it's observed. 

1317
01:09:51,818 --> 01:09:55,393
And, because again if I come 
back 
 to leadership, they see--

1318
01:09:55,393 --> 01:09:58,476
they, it's like the best 
intentions, I 
 see problem and 

1319
01:09:58,476 --> 01:10:01,150
I solve problem. 
And it's quick because you know 

1320
01:10:01,150 --> 01:10:03,606
things 
 need to be solved quick
and you're trying to make the 

1321
01:10:03,606 --> 01:10:06,010
world better. 
But you could spend a little bit

1322
01:10:06,010 --> 01:10:07,854

 more time being a little bit 
more intentional, understanding 

1323
01:10:07,854 --> 01:10:11,193
not just 
 where you see the 
problem, but actually what's 

1324
01:10:11,193 --> 01:10:14,223
kind of causing the problem. 
And if you're addressing where 

1325
01:10:14,223 --> 01:10:16,438
the 
 pain is caused, so much 
better. 

1326
01:10:16,948 --> 01:10:20,090
So it's not superficial, right? 
It's not just, that it's not 

1327
01:10:20,090 --> 01:10:22,802
just 
 there's not enough test 
coverage, it's that it's hard to

1328
01:10:22,802 --> 01:10:24,550
test, right? 
It's like there's all these 
 

1329
01:10:24,558 --> 01:10:26,998
kind of nuanced problems. 
And so that's the thing I would 

1330
01:10:26,998 --> 01:10:29,719
have, 
 I would stress is act 
where the pain is caused, not 

1331
01:10:29,719 --> 01:10:31,648
where it's observed. 
That's my first one. 

1332
01:10:31,698 --> 01:10:33,298
Let me turn it over to 
 
Elisabeth for her first one. 

1333
01:10:33,464 --> 01:10:36,212
Well, and I think my first one 
actually 
 dovetails really 

1334
01:10:36,212 --> 01:10:41,574
nicely with that, because it's 
about being very careful 
 to 

1335
01:10:41,574 --> 01:10:44,784
not conflate problems and 
solutions. 

1336
01:10:44,814 --> 01:10:47,024
And let me unpack that a little 
bit. 

1337
01:10:49,134 --> 01:10:53,626
Way back when I was a baby 
 
engineering manager and I 

1338
01:10:53,626 --> 01:10:57,080
believed that our problem was 
that we 
 didn't have enough 

1339
01:10:57,080 --> 01:11:00,536
unit testing. 
Well, no, that's… Unit testing 


1340
01:11:00,544 --> 01:11:05,534
was in my head the solution. 
It wasn't actually the problem. 

1341
01:11:05,664 --> 01:11:10,440
And so as a technical leader, 
being 
 able to not express a 

1342
01:11:10,440 --> 01:11:14,824
problem as the problem is we're 
not using AI, the 
 problem is 

1343
01:11:14,824 --> 01:11:18,759
we don't have enough testing, 
the problem… That's not the 

1344
01:11:18,759 --> 01:11:21,049
problem. 
That is your belief about the 

1345
01:11:21,049 --> 01:11:23,440
solution. 
And so drilling down to get to 

1346
01:11:23,440 --> 01:11:26,298
the 
 point where you can 
express the delta between 

1347
01:11:26,298 --> 01:11:29,954
observed, what you're 
 
observing in the world and the 

1348
01:11:30,024 --> 01:11:33,266
outcome that you're trying to 
drive 
 to, the outcome isn't 

1349
01:11:33,266 --> 01:11:35,914
unit testing, that is not what 
customers pay for. 

1350
01:11:35,944 --> 01:11:39,371
There is something else that is 
the 
 real problem, and you 

1351
01:11:39,371 --> 01:11:42,504
believe that unit testing is 
gonna solve that problem. 

1352
01:11:42,504 --> 01:11:46,013
I'm not saying I was wrong. 
I'm just saying that wasn't the 

1353
01:11:46,013 --> 01:11:47,830
problem. 
That was potentially the 
 

1354
01:11:47,838 --> 01:11:53,768
solution to the problem. 
And then a pi- a third one here 

1355
01:11:53,768 --> 01:11:57,924
for 
 you is be aware of the 
world of signals that you're-- 

1356
01:11:57,924 --> 01:12:01,479
that aren't there for you. 
It-- One of the… There's-- 
 

1357
01:12:01,487 --> 01:12:03,354
Look, I love the book. 
I'm biased. 

1358
01:12:03,814 --> 01:12:06,896
One of the things I think is 
wonderful in 
 the book is when 

1359
01:12:06,896 --> 01:12:09,880
we talk about dashboards. 
Dashboards tell you something 

1360
01:12:09,880 --> 01:12:11,726
happened. 
They don't tell you kinda how 
 

1361
01:12:11,734 --> 01:12:14,124
it happened or why it happened 
or what it took to get there. 

1362
01:12:14,554 --> 01:12:17,354
And so when I-- And so like, 
 
we need to have better signals. 

1363
01:12:17,384 --> 01:12:21,200
And I would tell leadership that
there 
 is this world of signals

1364
01:12:21,200 --> 01:12:22,920
around you. 
You just have to like kinda 
 

1365
01:12:22,928 --> 01:12:26,664
tune in a little bit, right? 
Kinda like it's not difficult. 

1366
01:12:26,774 --> 01:12:29,124
Like Elisabeth said, systems 
 
thinking is very approachable. 

1367
01:12:29,124 --> 01:12:32,134
I think people do it naturally. 
What I'd stress for leadership 

1368
01:12:32,134 --> 01:12:34,344
is to 
 be more attuned to the 
signals, right? 

1369
01:12:34,344 --> 01:12:37,714
It's-- Maybe it's the hesitation

 in the voice of the engineers.

1370
01:12:37,734 --> 01:12:39,944
That's a signal, right, 
 of 
something else, right? 

1371
01:12:39,944 --> 01:12:41,894
Or it's this meeting, like I 
mentioned, 
 that's happening 

1372
01:12:41,894 --> 01:12:44,858
over and over again, or there's 
all these little signals that 

1373
01:12:44,858 --> 01:12:47,798
are 
 happening that tell you 
that maybe there's a different 

1374
01:12:47,798 --> 01:12:49,958
problem that's happening, 
 and 
I need to understand a little 

1375
01:12:49,958 --> 01:12:52,454
bit deeper around what's going 
on here. 

1376
01:12:52,944 --> 01:12:58,161
Be attuned to the signals. 
And I guess I'll add to that, 

1377
01:12:58,161 --> 01:13:01,420
'cause 
 if we're talking about 
signals, let's also talk about 

1378
01:13:01,420 --> 01:13:04,597
levers, because it 
 is very 
tempting to reach for the levers

1379
01:13:04,597 --> 01:13:09,316
that worked in your last job. 
In my last job, we did this set 

1380
01:13:09,316 --> 01:13:12,892
of things, 
 and so pulling on 
those same levers, only you're 

1381
01:13:12,892 --> 01:13:15,820
in a different context now 
 and
those levers may not have the 

1382
01:13:15,820 --> 01:13:17,894
same effect that they had 
somewhere else. 

1383
01:13:18,704 --> 01:13:21,017
Furthermore, you may not 
necessarily 
 have been reading 

1384
01:13:21,017 --> 01:13:24,308
all the signals 'cause we only 
can see what we have 
 access 

1385
01:13:24,308 --> 01:13:27,758
to, and you may not have seen 
the consequences of some of 

1386
01:13:27,758 --> 01:13:31,101
those levers. 
And so being open to the 

1387
01:13:31,101 --> 01:13:34,577
possibility 
 that levers you 
never would've considered before

1388
01:13:34,577 --> 01:13:38,398
are in fact gonna be more 
effective 
 within your current 

1389
01:13:38,398 --> 01:13:41,088
context, being open to the 
possibility that there are 

1390
01:13:41,088 --> 01:13:43,144
levers 
 that you haven't 
identified yet that would 

1391
01:13:43,144 --> 01:13:46,844
actually be just as easy to 
reach as the 
 ones that you 

1392
01:13:46,844 --> 01:13:50,772
already know about, I think is 
really important for nav- 

1393
01:13:50,772 --> 01:13:54,674
especially 
 navigating in so 
much uncertainty and ambiguity 

1394
01:13:54,674 --> 01:14:00,692
in today's world. 
Last thing I might offer up is, 

1395
01:14:00,692 --> 01:14:05,684
 no problems are bigger than us.
And I think that's something 

1396
01:14:05,684 --> 01:14:08,933
that 
 engineering leadership, 
as well as even teams kind of 

1397
01:14:08,933 --> 01:14:11,950
need to hear. 
And I mean that in the 
 most 

1398
01:14:11,950 --> 01:14:14,553
kind way possible. 
Very frequently we'll get 

1399
01:14:14,553 --> 01:14:17,244
together with 
 groups and 
they'll say, "Well, you know, 

1400
01:14:17,694 --> 01:14:19,640
the environment is a bigger 
problem than 
 us. 

1401
01:14:19,640 --> 01:14:23,279
We can't solve it on our own." 
And it's like, well, but there 

1402
01:14:23,279 --> 01:14:26,664
are things that 
 within your 
control that you can influence 

1403
01:14:26,724 --> 01:14:28,574
that will shift it a little bit,
right? 

1404
01:14:28,574 --> 01:14:30,910
And so, that's where I think 
 
some of the beauty of the 

1405
01:14:30,910 --> 01:14:34,336
systems thinking comes into play
is that 
 we don't become 

1406
01:14:34,336 --> 01:14:36,889
powerless now. 
We understand kind of the things

1407
01:14:36,889 --> 01:14:39,604

 we can influence, and there 
aren't problems bigger than us. 

1408
01:14:39,604 --> 01:14:42,438
Because let's be honest, if 
engineering 
 leadership is 

1409
01:14:42,438 --> 01:14:45,214
always saying, "Well, the 
problem's bigger than us, we 

1410
01:14:45,214 --> 01:14:47,384
can't solve 
 it," those 
problems don't get solved. 

1411
01:14:47,544 --> 01:14:50,024
And so we have to be thinking 
 
about what we can do. 

1412
01:14:51,588 --> 01:14:53,078
I think that's a great place to 
end it. 

1413
01:14:53,178 --> 01:14:55,258
So there, between the 
 two of 
us, you got five. 

1414
01:14:56,472 --> 01:14:57,282
Nice. 
Nice. 

1415
01:14:58,672 --> 01:15:01,318
I love all the systems thinking 
wisdom 
 and the 

1416
01:15:01,318 --> 01:15:04,028
interrelatedness between both of
you, so I think really lovely, 

1417
01:15:04,028 --> 01:15:07,092
and it's a 
 very good food for 
thought for all of us. 

1418
01:15:07,772 --> 01:15:09,832
So thank you, Elisabeth, thank 
you, Joel. 

1419
01:15:09,852 --> 01:15:12,638
For people who would love to, 
you 
 know, learn more about the

1420
01:15:12,638 --> 01:15:15,372
book, maybe reach out to you for
questions, are 
 there places 

1421
01:15:15,372 --> 01:15:20,485
for them to reach out? 
Well, we have a website, 
 

1422
01:15:20,493 --> 01:15:22,214
signalsandlevers.com. 
The book is not officially 

1423
01:15:22,214 --> 01:15:24,638
released yet. 
It is available for pre-order. 

1424
01:15:25,358 --> 01:15:29,316
And signalsandlevers.com is 
currently 
 just a landing page,

1425
01:15:29,316 --> 01:15:32,858
but that is the best space to 
keep watching. 

1426
01:15:33,488 --> 01:15:35,787
We'll be building that out as we

 get closer to the release 

1427
01:15:35,787 --> 01:15:38,091
date. 
And on the Signals and Levers 
 

1428
01:15:38,099 --> 01:15:40,956
website, if you sign up, the-- 
we are starting to publish a 

1429
01:15:40,956 --> 01:15:42,515
newsletter. 
The newsletter has some 

1430
01:15:42,515 --> 01:15:44,811
behind-the-scenes 
 information 
as well as upcoming events. 

1431
01:15:44,811 --> 01:15:49,275
And Henry, I, if I told you 
there wasn't 
 a second book 

1432
01:15:49,275 --> 01:15:52,607
probably cut from the first 
book, I think we have like 

1433
01:15:52,607 --> 01:15:54,812
70,000 words 
 that were cut 
from this book already. 

1434
01:15:55,252 --> 01:15:57,971
So there is plenty of extra 
content 
 that will be dripped 

1435
01:15:57,971 --> 01:16:00,322
out as well. 
So if you're interested, 
 

1436
01:16:00,330 --> 01:16:03,563
please check out the website. 
Nice. 

1437
01:16:03,573 --> 01:16:06,453
So yeah, for those of you who 
love 
 these kind of topics, 

1438
01:16:06,453 --> 01:16:09,903
definitely check out, 
signalandlevers.com, right? 

1439
01:16:10,223 --> 01:16:12,033
And do subscribe to 
 the 
newsletter as well. 

1440
01:16:12,333 --> 01:16:14,703
So thank you so much for 
 both 
of your sharing. 

1441
01:16:14,723 --> 01:16:18,103
So really looking forward 
 for 
the release of the book. 

1442
01:16:18,423 --> 01:16:20,765
So good luck with the release. 
So thank you for your time 

1443
01:16:20,765 --> 01:16:22,632
today. 
Thank you so much.

