So in our book, we talk about 10 different aspects of quality.
Culture is one, and it's one of the top ones.
It's a social aspect, but testing is really such a small
part of the overall thinking about a quality culture.
You have to have the testing practice, but it's just a really
tiny part of it. I often see that the feedback
loops is the first one to get cut when they're feeling
pressure for time where they're too busy, so trying to focus on
getting that in healthy shape. I don't think that's a
compromise that's worth making. If you don't have a quality
culture, if you don't have good feedback loops, if you don't
have a good way to learn and improve without those three
social aspects, it doesn't matter who you aren't, your
technical aspects, your quality is going to suffer.
This is a question I get really often is we put defect
management at the very bottom right.
It really is the lowest priority because if you get all the other
things right, all of a sudden you don't have to worry about
defect management. It becomes a non issue.
We debated about putting it in at all.
At one point I was able to track in the organization I was in of
several thousand people. 40 to 50% of people's time was spent
on these activities, right? Just focusing on the defects and
managing them and tracking them isn't going to save you the time
and the money that you want to save.
If you measure the defects well, it tells you how bad the quality
is. It does not tell you how good it
is. Hello everyone, welcome back to
another new episode of the Technicianal Podcast.
Today I have one repeat guest and one new guest, Janet Gregory
and Selena Delici. So they just recently published
a book which I find it very interesting for us to discuss
and hopefully we can share some of the learnings from the new
books so that you can improve your quality practices.
So, Janet, Selena, welcome to the show.
Thank you very. Much.
All right, so since Janet has appeared in the episode before,
so maybe this time I would like to ask Selena to share any kind
of highlights or turning points from your courage and he so that
we can learn from you. Well, I started my career as a
software tester. Interestingly, back in the day
when I started, I did not know that was a real job, despite
going to school for computer programming.
Yeah. So I quickly moved into managing
and leading projects and teams. And one of the things that I
enjoyed most was kind of bucking the trends of testers sat in
their corner and developers sat in their corner and nearly did
we ever cross. And I was at least in the
company I was in, I was one of the early folks to try to bridge
that chasm and bring the programmers and testers together
to work towards finding solutions together.
And that worked really, really well.
And for the projects that I worked on with the testers and
the programmers, we were doing Agile Wiser, working without any
formal agile awareness, like we didn't know it was the thing at
the time 'cause this was in the early knots.
And then into the next organization I worked in, I
helped to bring Agile informally into the organization and said,
Oh my gosh, this was a lot of what we were doing, focusing on
collaboration and communication and testing ideas before I wrote
the code. And just found that it was
really great for everyone who was involved in the project.
And we were more successful with the projects that we were
working on as well that we from there, I shifted into doing
coaching and consulting around quality practices as well as
Agile. And I've shifted into doing also
executive coaching. So there's a myriad of things,
but the premise of all of the work that I do is really about
focusing on the quality of the environment and the quality of
the people. I do have the technical
background to back up what I'm talking about and be able to
speak with the technical people I work with, even though I don't
focus on that exclusively anymore.
But I found that when we focus on creating environments that
are sustainable and healthy for people, they're able to do
amazing work and all of the other benefits and gains that
the organization is looking for. Just follow suit.
And that said, having some of the technical experience allows
me to speak with really technical people who think maybe
I'm too fluffy for them. So, you know, there's a bridge
of being able to have all of the social, cultural as well as the
technical, you know, experience to kind of bring it into a
package. It really goes a long way for
everybody. Thanks for sharing your story.
I think hearing just what you said, it seems like quite
reflected in the book, right? The gist of the book, so talking
about quality practices and how the social part and also the
technical part coming in together into the team.
So maybe let's move to the book itself, right?
The book is titled Assessing Quality assessments using QPAM.
But the first thing that I want to talk about is about the
quality culture. So in your book, you basically
mentioned all these assessments is basically to assess our
team's quality culture. But in the 1st place, what is
actually quality culture like? How can we define it?
That's probably the first question.
It's funny because I've done a presentation on culture and
keeping an agile culture as such, but when you start to
think about what it means, it means the norms.
A lot of people act, do they do what they say, you know, walk
the talk kind of thing. So culture is about how an
organization works, but it's their ecosystem.
As such, when you're assessing A-Team, does that team fit into
that organization, or does it have its own norms?
Does it have its own rituals? Selena, I'm sure you can add to
that. Yeah.
I think just in terms of that quality aspect of culture to
enhance what you know was saying, it's how do we bring
mindfulness and insight and reflection into the work that
we're doing, whether it's writing code or writing a test
or understanding our requirements or just in how
we're communicating with the people around us, right?
And having that idea of I want this to be a quality interaction
or I want this to be a quality test and being thoughtful about
that really makes a big difference.
It seems the organization, the teams, the customers, a lot of
problems down the road, right? We're building in a quality
product because we have a quality culture.
I like the phrase that you use, right, the mindfulness and the
reflection of how you conduct stuff right.
I think many people associate quality culture with testing
culture. Is this the right thing to think
about or do you have any different perspective here?
The. Way I'm about it is testing is a
subset of the activities you would do starting to build in
that culture. I know in our book we talk about
10 different aspects of quality. Culture is one and it's one of
the top ones. It's a social aspect, but how an
organization, say, communicates, that's part of the culture as
well. Is it top down or bottom up?
Does it do some of both? How does it communicate?
So testing is really such a small part of the overall
culture or the overall thinking about a quality culture.
You have to have the testing practice, but it's just a really
tiny part of it. Well, maybe not tiny.
Yeah, yeah. And with testing, it's not just
like you're actually testing the software, testing the code,
right? It's we're testing ideas.
That's basically what it comes down to all the way through,
right. And when we can bring those
questions early on, we're helping to build that quality.
And as well, there's so much more that goes into that.
And you kind of touched on the the mindfulness phrase that I
use. I don't believe that a lot of
people would look at it that way.
Probably not. But I mean that is kind of like
the underlying thing that I'm watching for.
If there's thoughtfulness, that's going into the work that
people are doing, regardless of the particular task that they're
doing. Yeah, probably it's the
unconsciousness, right? Like people don't have this
mindfulness thinking about the quality practices that they have
and they always associate like with testing, testing, automated
testing, right? How much coverage do we have?
How many testers do we have? How many test cases?
So I think the point in your book that you try to bring is
actually testing is just one part of the quality thing,
right? And there are many other
aspects. And I think maybe in the 1st
place, what made you wanted to write this book?
So like, do you find it difficult to actually assess
people's quality practices? Like maybe what made you write
this book? We've both been asked to do with
husbands for organizations in our past and Janet was asked by
a particular client. So I think just better formalize
what that might look like. We came across a couple of other
models and Janet said, hey, would you want to help me build
something out? And then we suddenly had our
quality practices assessment model and said well, now what?
We think we have something of value to share with the
industry. So be hummed and hot and try
down some ideas and land it on. You know, the easiest thing to
do, which actually turned out to be more work than we thought,
was to write a book about it so that we can get it out in and
share it with, you know, the industry with our colleagues.
So that we weren't to bottleneck if people could just pick it up
and run with it as they wanted to.
But but is it common that many people poorly conducting their
culture or, you know, like withholding their quality?
And hence I think, should every team try to think about doing
this kind of assessment to actually know where they're at
and actually how to improve? I would like to see a really
team do an assessment like this. There's lots of them out there,
but if they know where they're at, then they can start moving
forward, right? If you never stop and assess
whatever level you want to be at or whatever, you can assess
small portion of what you're doing, for example, and say
where are we? And if you never stop and look
at that, then sometimes teams just start spinning their wheels
and get in a rut and really don't know what.
They try fixing that and fixing that, but they don't understand
why or where they're trying to even go.
Yeah. So in the past I've tried my own
assessment as well, my own version of assessment.
And actually it's really hard first is to define the model,
right or like the questions to ask.
And then after that you probably have different kind of stages to
actually represent where team at.
I think it's really hard kind of problem.
So I really thank you for coming up with this kind of assessment,
which I think is quite holistic. So maybe let's move to the
actually Q Pam itself, right, the quality practices assessment
model. So there are 10 practices that
you include in this model and also four kind of like stages
for people to actually know where they're at.
So maybe let's start with all the 10 different practices
first, like in the high level, what are those?
And yeah, maybe we can talk from there.
Yeah. So we'll call the top level
quality aspects. And then within each of the
aspects we have a number of practices identified.
So the quality aspects that we've identified, the tenor,
feedback loops and culture, learning and improvement, those
are the social ones. Development approach quality and
test ownership are the social, technical and testing breath
code quality and technical debts, automation and tools,
deployment pipelines and defect managements are the non social.
So those are the technical ones. So there's a lot in there.
And I think a lot of people would be surprised that like we
did a lot of work and we reached out to colleagues to kind of
prioritize them in terms of what are the most important ones to
focus on 1st to get your organization in better shape.
And I think a lot of people would be surprised that we don't
focus on the technical ones first, that we're focusing on
the social ones first. We start to get feedback loops
in place early on, for example, like within our tests, within
our code or feedback loops within our team, feedback loops
with our stakeholders, feedback loops for their customers.
That really just helps to smooth everything else out for the
other aspects down the road. What would you ask them to?
Yeah. How do we communicate those
feedback loops is really about how do we communicate with each
other? And if you don't have those,
then you end up with throwing it over the wall and a lot of
silos. So that was why we kind of chose
that number one. It's interesting, I often see
and organizing, 'cause I do consulting and coaching working
companies, I often see that the feedback loops is the first one
to get cut when they're feeling pressure for time where they're
too busy. So trying to focus on getting
that in healthy shape, I don't think that's a compromise that's
worth making. And that alone, focusing on that
and doing that really well is gonna help reduce costs, improve
quality, increase value, all of the things that organizations
want to have happen. At one conversation, it can take
away so many misunderstandings. I think intuitively when people
understand that software development is a social
technical kind of systems involved, right, they actually
understand that they should probably prioritize social
aspect more. You know, all these feedback
loops, communication, collaboration, psychological
safety and things like that. But actually many people do not
have this same understanding about social technical.
And that's why probably intuitively for them is actually
focus more on the technical practices first.
So things like, you know, CICD or automatic testing and things
like that. So I think I like in the book
that you actually sequence these technical practices in the order
of importance, right? And actually just Selena
mentioned you start with social and then some technical aspects
at the end. So maybe a little bit here, like
explain why do you think this order of importance is actually
very important? Well, so one of the things that
I'll be just say right off the bat, because this is a question
I get really often is we put defect management at the very
bottom, right, because it's really the lowest.
It really is the lowest priority because if you get all the other
things right, all of a sudden you don't have to worry about
defect management. It becomes a non issue.
We debated about putting it in at all, but we did in priority
as if you have, this makes the next thing easier, right?
Because if you don't have those social aspects, if you don't
have a quality culture, if you don't have good feedback loops,
if you don't have a good way to learn and improve without those
three social specs, it doesn't matter how good you aren't your
technical aspects, your quality is going to suffer.
The quality of the product, the quality of the process.
So that's kind of why we put those ones first for sure.
Do you want to take the other one Selena?
I think you kind of hit it on the hence on it like to your
point, you know, once you get the next things in place, the
next things go. Like if your development
approach is in good shape, then quality and test ownership can
more easily follow, right? And everything just kind of
follows through one after another with more ease if we're
getting them in order one at a time.
That said, there's no requirements for any team to
work through improvements in their team in that order.
They might want to focus on the areas that they're struggling
the most in also, right. But if you're really starting
fresh and you want to do a complete overhaul, like by all
means work through the priority that, you know, we felt was more
more relevance and importance, yeah.
And so we're just going to say that a team may take, for
example, development approach and just do an assessment on
that to start with. Yeah, I like the way that you
explain like if you focus on the first few practices first,
right, the remaining practices will become easier to implement.
I think this again, intuitively makes sense if you understand
this social technical aspect. But for people that also
struggle to actually thinking about how to improve this
quality assessment, I think this model is kind of like good,
right? But I have one question because
because typically if we find this kind of model somewhere and
you have different stages, they will call this maturity model.
Why in your book you specifically mentioned this is
not actually a maturity model. So maybe explain a little bit on
this. Neither of us are a big fan of
maturity models. That's probably the starting
point because it indicates that if you're doing things right,
you have to be at a certain state.
Like you have to go to the highest level, like you're not
doing a good job otherwise. And we don't think that's true.
We think that each organization and you know, and team is unique
and the environment that they're in and the desires that they
want for themselves, it should be respected, right?
So for one team like hitting, moving from beginning to
unifying to practicing might be enough in terms of their testing
breadth and development approach, for example.
And they say, know what, we're really happy here and we don't
need to go into the innovation stage.
It doesn't make sense for our product.
It doesn't make sense for our people and our customers don't
care enough. You know, like, it could be
something like that, right? So really it's about respecting
the uniqueness of each environment and the people and
the product and that they're able to make those decisions for
themselves. The key is being aware of what
possibilities might be there for them and really making a
conscious choice rather than just blindly landing somewhere
and saying we're the best because we're here, but we don't
know where here is and we don't know you know, what the other
options are, right? So we want teams to
organizations, ideally to make conscious decisions about what's
best for them rather than blindly landing somewhere.
Yeah, thanks for explaining that.
I, I like the way that you phrase their options that you
can aspire to become, right? And then you consciously make
decision rather than maturity model typically is and like
released by, I don't know, like consulting or some industry best
practices and management. Once they see it, they just say,
OK, we want to be the most mature organization ever, right
by following this maturity model.
Sorry, can I just add to that? Yeah, like you're you're hitting
on a hand like it's not a best practices, right.
It's options available and when people are striving to maturity
model, people will start to gain the system so that they believe
that they're high performing or world class when they're
actually not because they need to work good to management,
right? So moving against that is
important and so that people can be honest with themselves and
they feel safe to be honest with where they're at and where do
they want to land. And it's OK.
It's OK that they don't want to be at innovation, which kind of
goes in part with the psychological safety aspect of
culture, right? And if we can promote a model
that espouses that, I think that's healthier for everybody,
too. Yeah, I think the spirit is the
most important thing, right. So I find all these kind of
assessment typically is like what you said, right?
People will just game it because once you make it like a KPI,
right, you will just find the metrics, right?
What are the metrics that are being assessed and try to game
that and you know, and try to actually make people feel that
they're actually improving. So I think the spirit is very
important and we should not kind of like blame, right?
So you mentioned about psychological safety, like not
blame the people for the bad practices that they're doing
simply because if it works for their context, like, why not?
So I think you define the four different dimensions you
mentioned, right? It's not like maturity models
again. So there are four different
dimensions and it's kind of like the different terms that you
use. Maybe a little bit explaining
here as well. What are the four different
dimensions? So we landed on the idea of
beginning because we tossed around a lot of terms.
We asked them up a lot. We decided everybody knows where
they are. So beginning, it's kind of where
are you starting? And usually that'll happen.
And it still amazes me how many teams are still moving to Agile
that aren't they've come from waterfall, maybe they've come
from some chaos or something else.
Who knows. But the beginning, where are you
starting? You know that you have issues,
you want to get better. So we just said, OK, we're
beginning. And then the unifying is the
idea that the team is starting to adapt to agile practices.
Whether you call it agile or not, I really don't care.
But they're starting to think about collaboration more.
They're trying to get those communication lines better.
So we talked about unifying, they're doing the practices
they're trying to understand and then practicing this thing is
really our right. We understand we're delivering
our products on a regular basis, whatever regular basis means
we're not having very many defects in.
We're got a continuous integration and delivery cycle
going. We're practicing, we're doing
well and then innovating is those teams that, and I've been
on a few and they're such fun, but there's a lot of stress too,
because you're always pushing the edge and and you're
innovating. You work together as that really
great team and you're just constantly looking for what a
better yourself, right? And so that's innovating.
Did I miss anything Selena? Oh no, you did great.
And I would just add, you know, Janet said you could be agile or
we don't care what you call it. I mean, we've titled the book
Assessing Natural Quality Practices with QPAN, but it
doesn't have to be agile. I mean, the particular practices
embodied are just healthy practices for any software
technical organization, right? Irregardless of agile, whatever
comes after it, it doesn't really matter.
Like these are just healthy practices for any software
organization if we have them in place.
When I started my career a long long time ago, we didn't work in
Angel then, but boy oh boy, we would have been in better shape
if we had been replying this stuff right?
I think the same could be said now or 15 years from now.
Thanks for explaining all these different dimensions.
I think it kind of like makes sense, right?
People always begins when they learn about new practice, right?
So you have the beginning, you have then the unifying right?
Trying to gather the thoughts, adapting to the practices and
understanding actually the gist of the practices.
And then you move into practices, maybe when those
practices becomes a habit, right?
You kind of like do all the time and then innovating is always
pushing the boundary continuously improve from what
you understand. I think this is probably the
Suhari thing, right? So in Agile you have this also
loop, right? So yeah, I think I like this
dimensions. So maybe if we can, right, let's
just touch on a little bit on few of the practices to give
people flavor, how they actually use this quality aspects and
dimensions. So maybe let's start with the
most important one for sure, right?
The feedback loops. We kind of like touch on a
little bit here and there about it.
So feedback loops is also one thing that kind of like brought
up quite a lot recently about developer experience, about also
the flow things, right? And also continuous delivery and
all that, all gathers around feedback loops.
What is the most important thing here?
How can people assess their feedback loops against all these
4 dimensions? So we've evaluated that.
It was only tough to try to figure out practices or wrong
feedback loops. So we talked about within the
delivery team themselves, things like a lot of programmers and
testers talk to each other or anybody else on the team.
So in beginning they probably have, you know, throw it over
the wall kind of thing. They don't talk to each other.
They were lots of defects, things like that.
How are they actually talking to each other?
Then we have the idea of between customers, stakeholders and the
delivery team. How do you interact with your
customers? Do you even talk to them?
Do you listen to what they have to say?
Like if you were giving a Dunlop and I watch teams, then on teams
where you give a demo, the stakeholders lots of good things
and nobody writes anything wrong.
They don't pay attention. And then the third one that we
talked about is between leadership and the delivery
team. So in a beginning kind of thing,
it's probably a hierarchical 1 dimensional.
The leadership team says, hey, thou shalt do this and everybody
else goes running and does that changes priorities a million
times because somebody tells them to.
And so when you looking at those different aspects in those
particular practices, you can kind of see that as say a
practicing team will have multi directional kind of feedback
loops with their leadership team.
They are not afraid because they have psychological safety.
They're not afraid to say, hey, you know what, that won't work.
Can we try this and improve those kind of things for us?
At the beginning, you're really just doing what somebody tells
you to do, O. That's how we're assessing those
feedback loop on those 3 dimensions, those 3 ractices.
Yeah, I think feedback loop just like for any kind of leaders or
management, right? Maybe again like they are not
aware of about this concept, they've think it's fluffy,
right? What do you mean by feedback
loops, right? So I think it's not easily
quantifiable. So maybe for management of
people, when you want to explain, you should improve your
feedback loop because like feedback loop, I think it's
very, very evident. You know, in agile practices,
DevOps practices, this is probably the one thing that they
focus a lot on, right? But for management to actually
understand the concept of feedback loops, how would you
explain this? So think of like when you do
this assessment and you have to explain why feedback loop is the
most important thing. Well, it's all about checking in
and getting feedback, right? So basically we're testing ideas
basically as we're going, so you know, as a tester to a
programmer, a business analyst to a tester, let's say like
we're asking questions of each other frequently day-to-day
about going on and what's our idea and what's our design and
checking those assumptions as we go, right.
And then the programmer maybe say, hey, tester, come sit over
here. I have some questions about what
I'm doing and writing the code. Can you pair with me on that?
So we're constantly checking those things.
And it's the same with as we're communicating with stakeholders
or customers or management to the team that we are ensuring
that we're constantly checking in to make sure that we're on
the optimal path to shorten our cycle time and do that with high
quality so that our customers get high value.
So in the book, we've listed a lot of questions that could be
asked for each of the specific areas, and some of them are a
little qualitative, maybe a little fuzzier, Geely.
But how does a tester team member know what is ready for
testing? I mean, like, what is your tool?
What is your source? Is it a communication?
Is it a sticky note on her wall? Is it something on a task board?
Is it something that comes through where the code is
stored, like checking in on how do we know when something's
ready to test? Does the team document decisions
that they make Right. So some of them are yes or no.
And then where do we do that? It's really like there's a
really good comprehensive, holistic set of questions that
we've included. So really check in on.
And I think that the questions really get into the meat of what
it is that you would want to be looking for, for any of the
particular areas. But what it really comes down
to, I think if we're talking to senior management, like what do
they care about? Get it done faster, save us
money, make our customers happy. Like there's other things of
course too, but those are often the predominant forces.
And it seems a little counterintuitive that having
more conversations and more check ins is gonna get us the
faster throughput with higher value and reduced for cost.
But there's good data that you could find in your own
organization, never mind from other ones that show that that
is the case. Because if we go too far down a
wrong path, then you're actually increasing time, you're
increasing costs and, you know, and down the road maybe reducing
the value that your customer's going to get because you didn't
give them what they actually wanted or needed.
So those checkpoints are really helpful.
It's akin to like going on a road trip and you know, you have
an idea of where you are and where you want to go, but you
never check the map and you don't check the map anywhere
along the route to know that maybe you're 300 miles off
track. Like you want to check in and
know that you're actually in the right path.
And maybe that fits for people who drove without using AGPS
mapping tool back in the day like I did Once Upon a time, but
it's the same thing. Like pardon.
One of my favorite questions to ask to help them think about
what feedback loops is, is how does leadership get the
information they need to make decisions?
That tells you lots about feedback loops between
leadership and the delivery team.
That's what we're talking about, too.
I love that question. So actually that make you
reflect a lot about how do you get the current situation of the
team, right? Where the team at what kind of
struggle challenges that they're currently facing, right?
And I like the way Selena mentioned, right?
So it's not about feedback loop off delivering features fast.
It's actually testing ideas, knowing the direction, right?
Knowing that actually you are moving still in the right
direction. I think these are all very, very
important understanding that we have to have about feedback
loop. So I think thanks for sharing
that. So maybe the other quality
aspect that I want to touch on is about development approach.
So maybe when people talk about development approach, they look
at the developers, how are you writing code or maybe things
like architecture design. So maybe something more that you
touch on in the book, right? Development approach is not just
about writing code, so maybe brief us about this.
So development approaches practice the team and we call it
the delivery team. How do they work together?
How is the team made-up? Is it remote or is it together
like those things will? How the team works doesn't mean
there's a right of room, it just does.
So things like quality ownership, does the team really
think that the testers own quality?
And when we say delivery team, it's programmers, testers,
analysts, product owners, who is on your team and and how do they
do that thinking about what practices they do.
Like if you have a whatever agile point you're using will
depend on the practices, if you're doing Kanban versus Scrum
versus extreme programming versus whatever.
So there will be different practices and we don't specify
those specifically. We just kind of what's important
to you trying to understand what that is.
Do you have some kind of prioritization on your stories
or your features you're bringing in?
How do you handle those, those sorts of things?
Do you know when a story is done?
What does that mean to you? How do you work with delivering
it to production? How does the team work together?
So sorts of things when we're looking at a development
approach is right currently very beginning.
Do you have a mechanism in there for asking the questions right
through to delivery? How do you deliver to
production? What is that?
Yeah. So I think it's really important
that the team actually have a mechanism or the practices in
place, right? So first of all, it's not like
ad hoc, right? Like people just do whatever
practice that they think they want to do, but actually have a
method and approach, right? And then like you also
holistically look at, but just the code aspect, but also like
things like gathering the requirements, right, the
ownership about the code and the testing part.
And also like how do you release that story or features to the
customers, right? And then get the feedback loop
again back to the team. So I think development approach
here is really important because I find so many teams probably
like they just take agile framework, any agile framework
and they think that's the development approach, right?
Take Scrum. So they just say, yeah, we do
stand up retrospectives and you know, product backlog rooming
and things like that. And they think that's enough.
So I think the point here is that it's more beyond just the
code and the framework, but it encompasses so many other
aspects as well. So maybe the other quality
aspects that I'd like to touch on is about the defect
management. Like interestingly, Jenna, in
the beginning you mentioned that you actually thought about
removing this from the quality aspects.
But for people actually to assess quality normally, again,
intuitively, the first thing they they will do is how do we
track defects? How do we lock defects?
How do we know that they are closed?
So maybe explain why defect management is put as lowest
order in the priority. Yeah.
So with defect management, it's a why it's lowest out is because
if we're doing really well in the other areas, we've improved
in a way that the other areas are feeling really strong, then
amount of defects that we have should be going down
dramatically. So we will have less of a need
to track and manage those things.
Certainly, there may be some escapes, but we shouldn't be
going intentionally into releasing things with a lot of
known issues ideally, right. So we're doing really well in
the other areas. Our need for management is
reduced greatly. And one of the ways that we do
that is when we find things when we are in the process of
developing, which includes the programming and the
understanding and the testing, we fix them right away.
We don't consider things to be completed until we've addressed
the issues that we found right, in which case what we have as
defects out in the world are things that are being found by
people outside of the team in general.
And again, there should be a lot less if we're doing other things
really well. So from there, we're looking at
issues around how do we report defects, how do we triage them
and how do we report on them. Some organizations, they will
choose to use defect management tools.
Still, one of the things that we recommend that we've seen work
really well is that defects get prioritized right alongside with
new features. So we're making decisions and
we're storing those things right alongside with any features for
the products that we have in in the team's backlog.
So there's a clear decision about what's more valuable.
This may feature fixing what a customer has reported and that
in itself takes away the need to have a separate defect
management tool. That's one option that we've
both seen work really well. The time that's invested in the
reporting and fixing and triaging, I know early in my
career we spent a lot of time as testers, as programmers, as
project managers, as all of these different roles in
meetings and in sending emails and in dealing with the stuff
because there was a lot of problems, right?
And at one point I was able to track in the organization I was
in of several thousand people, 40 to 50% of people's time was
spent on these activities, right?
So it really, really, really is very helpful to focus on the
other quality aspects in doing them well so that you can get
that time back. Because just focusing on the
defects and managing them and tracking them isn't going to
save you the time and the money that you want to save.
You want to focus on fixing the other quality challenges you
have in your process within your organization.
And that in itself will address that and hopefully reduce it
dramatically, which I did see happen in organizations that I
worked in. Yeah, I'm only going to add one
thing, I think. If you measure the defects, what
tells you how bad the quality is?
It does not tell you how good it is.
Excellent close in color. Wow, that I think that's again
pretty deep, right? So you only know how bad you're
doing, but actually not how good you're doing.
And I think in your book you mentioned that the goal is
actually for defect prevention, right?
Not actually defect detection. I think many people try to
detect a lot of defects, right? Manage it, report that.
I think when you mention about reporting defects, I kind of
like left because in the past I also used to experience those
kind of things and not just just the report, but also the
fighting in between, right? When the testers report
something and developers don't agree, you kind of like fight
instead of prioritizing. So I think all this really
important because in many teams culture, right?
Still the fact management, you know, the Jira or whatever tools
that they use is probably the centre of their quality, so to
speak, right? And I think very importantly in
your book, you also mentioned that we should actually spend
more time in understanding the kind of problems, the features
that we're doing and the kind of testing that we want to build
from the very beginning, right? Not just something that you
threw over the wall or you think about it at the end.
So maybe a little bit on this part, how can people start
thinking about understanding the problem 1st and you know, like
building the test cases in the earlier phase rather than the
last? So I'm going to tell a story
here and people don't think about this necessarily as
testing, but I do it because, yeah, it's well.
But I was working with the team one time and they were all ready
to start on a feature. They were going to have their
kickoff meeting and just start going with this new feature.
Product owner had done a bunch of work.
They had stories ready to go and I had been trying to coach them
a little bit on maybe let's start a little earlier thinking
about some of these things. So I said before we do that, can
we just take time due to the hence an hour and let's build
the mind map. The product owner understands
this feature really, really well.
The team hasn't seen it so much, so they'll be learning about it,
but asking questions, trying to understand.
So can we just have a quick half hour meeting and build a mind
map on what this feature is? And so, you know, the product
owner was a little, I'm not sure, but he agreed and the team
said, OK, we're they're willing to try something new.
So we had this half and we spent half an hour building a mind
map. It was remote, we did it online.
Before we finished that half hour, there were so many
questions that had come up from the team build it, bringing up
risks, asking questions that had not been thought about before
that they actually took that feature off the table and said
we're not ready. So that half hour prevented
probably weeks and weeks and weeks of blocking stories trying
to understand this. They just took it off the table
and went back and started asking some of those questions.
That to me is testing early. We're testing our ideas, we're
testing our understanding, we're exposing risks.
It's just not testing the software.
I think that's a really powerful story that we can learn from,
right? So I think this also comes back
to the holistic testing approach.
I think we covered that in the previous episode as well, right?
So testing is not just testing the code, testing the product,
the feature, but also testing the risk, the ideas, right?
And make it early, as early as possible.
I think it's a very, very important.
So I, I really love the story, right?
So I think it's quite typical in many software teams, right?
Some product owners or stakeholders will just say, OK,
build me this feature like in a few sentence, maybe better if
they have something, I want something.
So that's something, right? But it's always not enough.
They seem to miss the details, the edge cases and maybe the
variance of how the user journey will look like by using those
features. So I think thanks for raising
that. So knowing all these quality
aspects, then the next important thing is actually to conduct the
assessment itself, right? For teams who probably don't
have expertise, they cannot hire coaches like yourself.
How would you advise people? Because I know that you are also
writing the second book, right, To actually help people
facilitate this kind of assessment.
Yeah. So in the first book, the
Assessing Agile Quality Practices, we do have a chapter
devoted to how to conduct a quality assessment.
It's a brief synopsis of what we would recommend and our second
book, A Guide for Facilitating Quality assessments goes into
super detail. Everything that we would
recommend, whether you are an external consultants, assessor,
facilitator or internal, or you are a team member and there's
actually a whole chapter devoted to I'm a team member.
How do we want to do this as a team?
So the steps are gather information.
How do we prepare for the assessments?
Then we actually conduct the assessment.
We gather everything that we need to do and then how do we
put all the information together in a way that is meaningful and
assess all of the information that we pulled together.
And then how do we want to report on the results for the
team's benefit, for the organization's benefits and then
any follow up activities that you might want to do.
So there's a lot involved with each of those steps, whether
it's how do you identify who do include in the assessment?
Do you have examples of like stories of tests of code maybe
that you want to look at? Do they do things like mind
mapping? If so, like see examples of
that. I don't know if there are
examples. I used to work for a company who
we did do that sort of thing. And I know what Jana does too.
But that's not true everywhere, right?
But if they have examples, you want to see that and setting up
all of the meeting invitations, getting the right people in the
room, whether you're in a room physically or doing it
virtually. And there's also things like
doing one-on-one discussions with people, like a selection of
subset of people or you're doing group interviews with say maybe
all of the testers or all of the managers who are involved or
whomever it makes sense to have those discussions with.
But there's a lot of sources that we're pulling together.
One of the really bigger activities that we would do in
terms of conducting is what is called a process retrospective.
So understanding from the very beginning of our project that we
work on all the way through to delivery, what are all of the
steps and stages and activities that take place to understand
like how it all fits together with all of the people involved
in the room together. And often times you will see
light bulbs or questions or confusion pieces go off around,
I need to know that what's happening or what do you mean
that's what you do next. Like it's a really great way for
the whole team and the project team or the delivery team to get
on the same page about what is actually going on because they
don't often all know that. Right?
Yeah. And then pulling it together
into reports. We actually provide a tool that
you can do that with. For those who've purchased the
book, they can get the link. It's a free downloadable
spreadsheet that you can use to help you to do your assessment
and analyze your results. So yeah, for us it's like, yes,
we can do assessments. I know Janet's not anymore,
she's in her retirement case. But like myself and her other
co-author, Lisa Crispin, and like anyone who's picked up the
book, like frankly, you can pick it up and run with it and be
able to facilitate an assessment.
But you're more than welcome to involve people who have done
these before too, to learn from others who've done this a few
times. I like the way that you when you
describe a process retrospectives, right?
So sounds like a value stream mapping by the way, right?
Maybe it's kind of it's a little bit.
There, it's similar. Similar, yeah.
Yeah. So it's always a very revealing
session where people start to understand, oh, what do you mean
that we have this or what do you mean we are doing those kind of
stuffs that are unnecessary, right?
And it's not just for the team, right?
The leaders also, they sometimes don't actually have a good
understanding what the team is actually doing and what are the
process involved or maybe those kind of habitual things that are
probably legacy or bureaucracy. So actually they may not even
need to do that anymore, right? You can take the decision to
actually remove that from the process.
Maybe any kind of interesting letting.
I know that probably you have done this assessments a few
times with your customers. Any kind of revealing insights
or revelation from your customers after they're doing
this kind of assessment? Maybe if you can share one or
two. I think one of the biggest
things that I've seen the most often actually is the
programmers saying, I didn't know that you did all of those
things because testing is always, it seems to be one of
those hidden things. It's just magic.
And so when you start putting up all of the little things that is
being done, you will get a lot of that.
I just didn't know we can help you with that.
That's what you want to hear, right?
Yeah. I mean, I've heard that one too.
Absolutely. I mean, the biggest aha really
is just having it all there together in one visual picture.
It is really just a big aha for everyone in the room about what
it really takes to do the work that we've been asked to do.
That sounds like it's just, oh, just put this change in really
quickly, la, da, da, And it's all beautiful, right?
It's not that simple. It's not that easy.
And when we know all of the steps that are involved, yeah,
there are ways that we can find a way to accelerate in healthy
ways some of those things by automating the right things at
the right time. But we have to be mindful about
how we build quality into those steps too, right?
So it's not just we snap our fingers and it's out the door,
right? Even with things like bringing
AI in into organizations now, like there are things that can
be beneficial and helpful, which I mean, we didn't touch on in
this book, but it's the same, it's automation, right?
How do we bring automation and to support us and help us?
And there are considerations that we need to make to ensure
that we're making good choices. So regardless of how we look at
the process and see how big it is and how involved the news and
how complicated and all of these lines of communication and
feedback loops that need to happen all over, ensuring that
we are front loading and dressing quality rather than
leaving it to the end is is really what we want to see
people doing so that they are able to be as effective as
possible. And in my experience, an
activity like this helps people to start to make that connection
and understand how to get there 'cause now that they've looked
at it, they've had an assessment, they've gotten their
results back and they're like, oh, OK, we're really struggling
and like development approach testing rats and feedback loops.
Let's start with feedback loops first and then maybe the other
ones will kind of domino effects down right?
And then they can make conscious choices about that, but like
having those results and really understanding where they are and
why they're having the problems that they're having.
Yeah, that's the big AHA that I see with teams and
organizations. And I would say this management
in particular, because they often don't understand
everything that's involved. And this gives them something
tangible to hold on to, to understand how they can get from
this unknown like chaotic. I don't know why we are here,
but we're still consistently not doing a good job into This is
why we are here. And these are the steps that we
can take to get to where we want to go.
I mean, there's a lot of ideas, but all of them, there's a few
key things there. I hope that's too dope.
Yeah, I think it will be really cool if every team has this kind
of assessment, You know, they put together all these kind of
quality aspects where they are at.
I think, again, a lot of teams probably don't even understand
where they're at, right? And putting this together,
putting it as a map, right. And I think the most important
thing also like the road map, where, for example, if you're in
the beginning stage, you know, there's a road map that you can
aspire to, to become the innovating dimension, right?
So I think having this kind of road map that you put in place
for a continuous improvement process within the team will be
really, really something that is cool.
I tried to build that. I know it's probably fail more
than a successful one, but I think, yeah, it's really hard to
come up with this kind of assessment model.
And I encourage people to check out Janet's and Selena's book,
right? And maybe try to use that to
assess where you are. So we reached the end of our
conversation. But before I let you go, I have
one last question. Janet, maybe you still remember
last time I asked this. I call this 3 technical
leadership wisdom. So think of it like the last
advice that you want to give to the listeners.
Maybe if you can share the version of three technical
leadership wisdom from any of you.
OK, I'll give one because it comes right out of what we just
said, what Selena just said, visualize.
Visualize, whatever, whenever you possibly can.
Because for example, if you can see a process, then you can
discuss it and decide where you want to take it from now.
So I'm a big fan of visualizing and trying to draw a picture of
it, where there's a mind map, a flow diagram, whatever makes
sense. There's so many things that we
could add, I would say, and working with reality I think is
really important. And the tools, the model and
some of the tools that we talked about around assessments,
including visual, like visualizing, but where you are,
like if you just got dropped out of a plane with a blindfold on
and you don't know where you are in the world, you want to figure
out where you are before you make plans about what to do
next, right? And it's the same within the
context of your organization and the teams and your products,
like taking sometime and it feels like a slow down and
you're too busy to to do that. But taking that time to know
where you are, know your options for getting to where you want to
go is really important. And I see a lot of teams and
organizations struggle to do that.
And that's why consultants and coaches have jobs because we
help facilitate that from assess to understand where you are and
where do you want to go next. So yeah, that's really important
to do. I'm not going to add a third
one, which I'm sure Selena will agree with.
Don't be afraid to question, especially if something is
making you uncomfortable or is pushing your own ethics, your
own morals. Sometimes it's it's just
something and you're scared to ask that question because you
think it's a dumb question. But often that's the thing that
can make a team just stop and say we never thought of that.
So break out of your comfort zone and ask the question.
One reframer that is like of courage, but it's not even that
it's, you know, having the willingness to sit in the
uncomfortable space that you're in and sitting in uncertainty
and with the questions that you're asking.
Don't ask why. Questions go a little deeper
than that. Yeah, it kind of like also shows
vulnerability, right? I think Brené Brown has this
concept, right? So the courage is actually the
courage to actually be vulnerable, right?
Show vulnerability and ask these kind of questions, right?
So I think it's really important in a team where you have a lot
of psychological safety, this will be probably a key thing
where you can start being a high performing one.
So really beautiful technique wisdom.
So for people who want to check out the book or they want to
learn more, any kind of resources that you have
available online. Yeah.
So we have both books are available for purchase on Lean
Pub. So you can look up our names or
QPM, both books will be coming up.
So assessing actual quality practices with QPM and a guide
for facilitating quality assessments.
Both books are also available for purchase on Amazon worldwide
as a physical copy. So if you want a physical one,
that's where you can go buy that.
Otherwise website is forthcoming.
That is their focus for the next few months.
And yeah, otherwise you can find more about me on LinkedIn.
That's Selena, Gillespie and Janice.
Find me pretty much your name wrong.
My website is Janet Gregory ca.ca.
So yeah, from there you can find me anywhere, but if you Google
me I usually come up pretty quick, so.
Yeah, I can attest to that. So your name and Lisa Crispin
are always like the thought leaders in the testing space.
So I'll make sure to put all those resources you mentioned on
the show notes, right. Thank you again so much for this
conversation. I hope people learn a lot of
things about quality aspects and the dimensions and people start
to assess where they're at so that you can improve from where
you're at to something that you can aspire to become in the
future. So thanks again for your time.
Thank. You very much for having us.
The.