Hey, a quick message for those of you who are listening to this
episode on Spotify. I have a small favor to ask.
Spotify now allows mobile users to rate podcasts.
I would really appreciate it if you can take a quick pause to go
to the Technically Journal podcast page and leave your
favorite show your best rating on Spotify.
It will help me a lot to get this podcast to reach more
people on the platform. Thanks a lot.
Which is a good segue for us to start talking about your book
Effective Software Testing. I think one key sentence that I
pick when I read your book is that you mentioned to be an
effective developer, you must become an effective software
tester. But in practical world,
sometimes developer and tester, they are two different roles.
So tell us more about this. Why do you say that to become an
effective software developer, you need to become an effective
software tester? Thanks for the question.
I think it's because as a developer, it's your
responsibility to make sure that what you do works, and the only
way to do this is by writing tests that prove to you and to
your team that your code works. So indeed, before and I even
worked in environments like this where I would just focus on
coding and then I would send my software to another company.
This company would do the testing for me, they would send
me a report, they would fix the bug, so on and so forth.
And that's just not productive. Back and forth between these two
teams is too big. And I think, again, it's our
responsibility to make sure that things work.
And automated testing is just such an easy and cheap way of
doing it. And that's why I say in the book
that effective developer is someone that effectively tests
what they do, right? And I think, I mean, I don't
know, I was a developer last time.
Sometimes developers can be very confident type of person, right?
So we rewrite the code, we test a little bit during the local
development, we think it works. We always think it works.
Many developers probably do not enjoy writing tests for some
reasons, maybe apart from unit test sometimes because it's very
close to their workflow. So how would you actually invite
more software developers to actually have more passion or
energy to start writing tests? If they have the perception
that, yeah, I'm super confident of my code, I don't want to
spend more time to write automated tests and things like
that, yeah, that's a very interesting question.
So the first part was you mentioned about developers being
confident, right. And what I always say is you
never know. You don't know how many times a
day you make a mistake. So you program something wrong.
And you only know this once you have tests.
Because if you don't have tests, everything that you code looks
perfect, right? You just believe it works.
As soon as you have tests, then you see, you know, Oh my God,
I'm breaking my tests 50 times a day.
And then you quickly realize how much you need them because we
break stuff all the time. Now, how to convince developers
to write tests? That's a very deep question.
I think there are many angles to this.
One is to show to them and to have them practice so that they
don't see writing tests as something that is a burden.
They just get used to it. They get proficient with writing
tests, not only with the testing frameworks, but on writing code
that is easy to be tested on. You look at a program and the
test cases emerge in your head, and all you need to do is to
code them in your programming language.
So there's this aspect of just practicing and getting better at
tests itself. The second one, if you are in a
very large and complex software system, things are too complex,
right? And then for you to have this
pleasure back in writing tests, you need to make sure it's easy
to write tests. And this means as a company or
as a team, you have to invest in a small infrastructure that
facilitates the act of writing tests.
I was discussing this yesterday, actually, not here at the
company, but in another university that I was yesterday.
And I was saying, if you're working on a very complex
software system, to test something, maybe you need to
know to instantiate 10 entities or have data in 30 different
tables in your database. And that's just because your
business is complex and you have to have some something in there
that makes this easy for you That in a bunch of couple lines
of code you can set up the scenario you want to test and
then you can do the test. So those are the two
perspectives I see. One is again getting good at
testing itself and the second one is having the proper
platform so that you can write the tests.
Totally makes sense for me. So from my point of view also
sometimes we also need to maybe introduce back like what you
mentioned in the beginning, the pride, right?
So how can you tell that your code works now and also in the
future? There's no other way to do it
other than having some automated tests that can actually prove
that your work is always past, right in terms of test cases.
And I think many developers also think probably writing more code
means like doubling their effort, right.
So I think this must be changed as well in terms of perspective,
so that you don't finish writing the code if you don't have the
test. Maybe just some perspective from
me, Let's move to the thing that you mentioned in the book,
right. So will you mention about
effective? And there's another keyword that
actually you mentioned in the book which is systematic.
So I find these two things are very interesting the way you
explain in the book. Maybe if you can.
Explain here as well what will be your advice for people to be
a more effective software tester and also to be.
Systematic. In coming up with those tests,
yeah, indeed. So they make a very strong point
that maybe you can be a little bit more systematic when we
approach testing. We wrote a paper a couple of
years ago, and in this paper we asked developers to write tests.
So we gave them a program and write tests, and we observed
them and they were thinking aloud so that we could see how
they were coming up with test cases.
And we observed a couple of things.
Maybe the most important one for this conversation is the
developers were never systematic.
They were always, you know, just following their hearts.
They would look at the code. This feels like the next thing I
want to test, they would go and they would write the test and
then they would repeat what is the next thing I want to test.
So they would just follow their hearts and follow your
experience. And feelings are very important
when it comes to testing, but it also opens up space for
mistakes. Maybe you're not in a good day
and then you forget something. 2nd, if you're always using all
your brain power to come up with test cases, even the basic ones,
you're not saving energy to think of complex cases.
So when you're more systematic in terms of testing, that means
you know you just follow some sort of cake recipe that helps
you to come up with a bunch of test cases that are quite easy
to see. You can put them on a checklist
basically, and then it is your brain power to focus on test
cases. That really requires human smart
people thinking about it. And the fun part is there are
lots of techniques like this If you look at academic books on
testing, even the books from the 80s.
Bigger goal back then was to come up with a recipe that
teaches you how to write perfect tests, and there are lots of
good ideas there and you can think of.
I'm not going to give a tutorial here, of course, but you can
think of basic things like, for example, if your method receives
a list as an input, there are a bunch of interesting cases that
always makes sense. So for example, what happens if
the list is empty? What happens if the list is new?
If there's no linear programming language, what happens if the
list has just one element, right?
Because sometimes your algorithm changes if there are multiple
elements or just one. So there are a couple of basic
stuff that you don't even have to think and you can write those
tests and they are usually good enough to get you to a very high
coverage already and then leaving space for you to then
focus on boundary testing, you know, corner cases and this type
of stuff. And being systematic is
something that you see in other engineering fields, right.
And I think I cited in my book, there's this very famous book
called The Checklist Manifesto. And there's a story there backed
up by research that shows that medical doctors that use a
checklist before some surgery, they make less mistakes that
medical doctors that don't. And you don't have to be ashamed
of following a checklist. It's goods.
And I think that's the point of my book when I say we can be a
little bit more systematic. Of course we don't have to be
systematic all the time. It's just too expensive, maybe
too complex to be systematic all the time.
But identify, you know, like I'm testing a complex method here,
let me be a little bit more systematic.
So that's the whole idea of being systematic when it comes
to testing. Yeah, I think your point is
right. Sometimes we have typical use
cases, just like list or maybe you'll have an API, failure
cases, success cases. So all this can be made as a
checklist. I've heard about Checklist
Manifesto as well. I think the author named Atul
Gawande. And I think becoming systematic
in solving this kind of a testing problem is really
crucial because in your book you mentioned that if you are
systematic, you can assign any developers to the same problem.
They will most likely come up with the same test suite, right?
How cool is that? Because sometimes we feel that
we only need a certain testers to come up with a comprehensive
amount of number of tests. But I think the systematic way
is actually to have any developers working on the same
problem but they can come up with the same test suite.
That I think is really really powerful concept.
And I think it also becoming effective means that you write
the right test, because I think it never ends right.
You can come up with any number of tests as long as you can
produce the test inputs right. And I think to come up with a
certain right amount of test is very important.
Which brings us to some of the techniques later on.
One of the techniques that is commonly, I don't know, discuss
in the software world is about. Test pyramid maybe?
Let's start from there first. Do you think this is a
systematic way to think about producing tests and what?
Is your view about? This testing pyramid.
I think the testing pyramid helps you to be pragmatic when
writing those tests, because one thing is to come up with the
test cases, right? So those are the inputs I want
to give to my program and this is the expected output.
The other one is writing this in code and making sure that this
works nicely in your, let's say, development process, in your CI
and etc. So for example, if you just
write end to end tests, maybe your test suite will cost you
too much to execute and then at some point this becomes a
bottleneck, right? So I think the testing pyramids
gives this pragmatic point of view on, hey, we're going to
have to automate this test and we're going to have to maintain
those tests and we're going to run them the whole day.
And I like the idea of the testing pyramids very much.
And the idea is that at the bottom you have unit tests,
right? And why is it at the bottom?
Because they are usually cheaper to write, they are cheaper to
run, they tend to be more robust, they tend not to really
fail, they are not flaky in general.
And then you go up the pyramid, you have integration and end to
end tests and you still have to do them right.
You have to, but you do them a little bit less.
Maybe you focus a little bit more in the unit test, maybe
it's OK to maybe have some duplicated tests here if you're
unsure. Are you covering the same thing
or not. I think it's OK to make this
sort of mistakes in the unit test, but it's integration and
end to end test. You're going to prioritize a
little bit more and I like this idea of the pyramids.
Although you know in the software engineering at Google
book I think from 2018, I think they bring a new perspective
that I also appreciate very much that is forget about unit
testing and integration testing, right, just separate your test
suite between. Is this one fast enough that I
can actually run it during a pre merge together with your merge
request or pull request or is it super slow that I will have to
run in a separate machine and etcetera.
So separating the test suite in fast and slow makes a lot of
sense to me. Because if you remember back in
the 2000s, our big discussions were what is a unit test?
If I'm testing two classes together, is this 2 unit tests
or not? Right?
And I think in 2023 we know that this doesn't matter at all.
As long as it runs fast and it gives you proper feedback,
that's good. So I like this new way of seeing
things and it brings way less space for this type of
discussions, right? Because I feel no one can argue
that a slow test is better than a fast test.
You can argue that maybe like integration tests more than unit
tests, but no one can say a slow test is better than a fast test,
right? So I like this new way of
phrasing this. Yeah, let's go to the point you
mentioned just now, right? Because they were asked some
discussions and debates about people preferring more about
integration tests rather than unit tests, some think that
doing unit tests because you probably most of the things they
feel is less valuable. So what is your view about this?
I know it's it's probably a little bit more hot topics to
discuss, but what is your view on this?
I think I have a pretty clear opinion on this, right?
And I think mocking can be dangerous for sure, because if
you mock too much, then at the end you're not testing anything.
So you want to test as much as possible real behavior or
behavior that will look like in production.
But at the same time, you don't want to be slowed down by things
that you don't control so much. For example, let's say you're
writing a piece of code that makes a call to web service that
is developed by another team, and suddenly for you to really
run this test, you need that web service available to you, right?
And this becomes very quickly a pain.
So in this case it's mocking makes a lot of sense and the web
service you don't control so much.
Let's give an example of something you control a
database. Should I mock my database or
not? To be honest, I believe my
opinion is that you should not mock your database, because
today you can make tests with database so fast that you can
actually run them during your pre merge time and you get
feedback very fast. So why would you mock something
that is not really preventing you from writing the test in an
easy way and to run it fast. So I think that's the trade off
that needs to happen. I think it's not about writing
integration tests or unit tests, but it's about writing lots of
tests that are fast in the end and that you can have full
control over. And those are the things you
should mock, right? Things that you don't have
control over. And then of course, if you go
back to the testing pyramid, let's say you're mocking this
web service. So you can write lots of unit
tests, you know, super fast tests that just mock this web
service, but you can still have one or two or three integration
tests that are a bit more expensive that make real calls
to this web service. So you see that things work when
you put all the components together.
But you know something that I always say is you should not
write an integration test that could have been a unit test,
right? You don't need an integration
test to exercise an if statement in your code.
Just write a unit test for it. Leave integration test for what
it really pays off, that is to find integration bugs.
I really love your pragmatic approach to answering this
question so I fully. Agree as well in terms of
control, right. So first think about do you
control that and especially if not just the.
External service or infrastructure.
Related time is also probably one thing that you want to
consider. Right time is something you
cannot control by itself, but you can actually create like a
fake system or mock it sometimes.
So I really love the way you explain about this.
So let's move on to maybe some of the other techniques right
where you have talked about unit tests, integration tests, end to
end tests. In your book you actually
mentioned couple of testing techniques.
So let's start with the first one which is called the
specification based testing. And it's actually very
interesting the way you start by expanding this.
I would assume like some people would start with unit test, but
you start with specification based testing.
So maybe tell us more what is specification based testing, Why
you prioritize that as the first?
Cool, Yeah, So what is testing right?
So testing is about trying to find bugs.
And how do you do this? Because you compare what you
expect the program to do with what the program does, right?
And for you to do this you need to know what the program should
do. Where is this information in the
requirements? And I'm not saying like a Word
document that contains the requirements or UML, it doesn't
matter. The requirements can be in your
mind at some point. There's this notion of what the
program should do and specification based techniques
are the ones that help you look at the requirements and identify
interesting test cases. This is not new.
Again, this dates from the 80s and we became very good at
coming up with techniques. And basically what I showed in
my book is to look at the requirements.
It's quite easy to see those are the inputs of the program.
This is sort of what the program needs to do in the output.
Those are the different paths that the program may take, so on
and so forth. And you can look to all of this
and then get inspiration to write your tests.
And the one technique I show in my book is actually a very basic
one that basically says look at the inputs of your program, or
the method you're testing, or the component, whatever.
What are the inputs? Separate them one by one.
Let's say your function receives 3 inputs, an integer, a string
that represents whatever, and the list.
Look at them separately and explore their domain.
So let's say this is string. What are the possible strings
that can come here? Are there any special strings
that would make the program to do something different?
You do this per input. And why you do this separately?
Because it's just much easier for our brains to process small
things, right? So you do each one of them
separately, Then you try to look at all of them together.
You look for other possible corner cases and so that might
be explicit on the documentation or the requirements.
And then and only then you come up with test cases.
And that is sort of the idea of specification based testing,
that you start your tests from what the program should do and
not from the implementation. And I actually like this very
much because as a developer, right, the person writing the
codes that I'm about to test, that gives me the opportunity to
disconnect from my implementation and really focus,
hey, this is the input that I'm going to pass through the
program. This is the expected output.
So this is my suggestion. If you really want to be a
little bit more systematic when it comes to testing is first
step is start writing tests or creating test cases from the
specs from the requirements. So when we talk about
specification based testing, I think people always, I mean not
always often associate with BDD right Behaviour driven design
and sometimes it's like Gherkin type of language and things like
that. What's your view of this?
Do you always have to come up with BDD type of specification
test or is there some other types of tests?
That we can also use. Yeah, very good question and
maybe that's the difference between my book and other books.
Usually when people talk about specification based testing,
they are focusing on a very coarse grained feature from the
point of view of user what should I test?
In my book I show that you can do this at method level for
example, or at class level, because as a developer that's
sort of the units you're always handling with, right?
And I think to me BDD makes more sense when you're looking at the
big picture and looking at the whole functionality from really
the point of view of the final user and the specification based
techniques you can apply in both.
Now should you write tests in a BDD style?
I think that's really a matter of taste.
I am personally not a BDD fan, so I don't like tests using BDD
style. I don't use Cucumber, that's
just not my thing. But I see where this comes from,
right? And I don't think you know in
practice this is too different. It's really telling you to focus
on the specs, on the behavior of the program, right?
And coming up with the test cases.
Tooling is a matter of personal taste in the end, right?
As long as you're looking at the behavior of the program, what
you expect from it, I think this is a great step towards a good
testing. I hope you enjoyed this short
clip from Techly Juno favorite playlist.
If you find this episode useful, please help share it with your
friends and colleagues who you think would also benefit from
listening to this episode. And if you want to listen more
from this conversation, please go back and listen to the
original full conversation with my guests.
Stay tuned for the next Techly Journal episode, and until then,
goodbye.