1 00:00:00,000 --> 00:00:02,460 So we constantly have to keep asking ourselves 2 00:00:02,460 --> 00:00:05,240 regularly, like what does quality mean to our end 3 00:00:05,240 --> 00:00:08,060 users, to the people that matter, and what does it 4 00:00:08,060 --> 00:00:11,080 matter like over time? The quality is connected to the 5 00:00:11,080 --> 00:00:13,180 risks and the risks is connected to the testing. 6 00:00:13,740 --> 00:00:15,540 If we don't keep on our own 7 00:00:15,540 --> 00:00:19,100 quality, our testing and our development will drift 8 00:00:19,580 --> 00:00:21,740 because we are no longer building the thing that the 9 00:00:21,740 --> 00:00:29,160 people that matter care about anymore. The 10 00:00:29,160 --> 00:00:33,980 Show Hey everyone, my name is Henry Suriawirawan, and 11 00:00:33,980 --> 00:00:37,240 you're listening to the TechLegional Podcast, the 12 00:00:37,240 --> 00:00:39,280 show where I'll be bringing you the greatest technical 13 00:00:39,280 --> 00:00:42,620 leaders, practitioners, and thought leaders in the 14 00:00:42,620 --> 00:00:46,220 industry to discuss about their journey, ideas, and 15 00:00:46,220 --> 00:00:49,980 practices that we all can learn and apply to build a 16 00:00:49,980 --> 00:00:52,920 highly performing technical team and to make an impact 17 00:00:52,920 --> 00:00:56,280 in your personal work. So let's dive into our 18 00:00:56,280 --> 00:01:02,900 journal. Hello guys, welcome back to 19 00:01:02,900 --> 00:01:05,260 another new episode of TechLegional Podcast. Today 20 00:01:05,260 --> 00:01:07,300 I have Mark Winteringham here. So 21 00:01:07,300 --> 00:01:10,240 we're going to cover a lot about testing web APIs and 22 00:01:10,240 --> 00:01:11,960 also testing in general. And 23 00:01:11,960 --> 00:01:14,720 later on we'll sneak in the AI topics part as well 24 00:01:14,720 --> 00:01:16,960 because I think it's kind of like the recent trending 25 00:01:16,960 --> 00:01:20,160 topics, AI and testing. So Mark, thank you for coming 26 00:01:20,160 --> 00:01:23,180 and looking forward for this conversation. Thank 27 00:01:23,180 --> 00:01:24,920 you, thank you for having me on. Right, 28 00:01:25,020 --> 00:01:27,440 Mark, I always love to start my conversation by 29 00:01:27,440 --> 00:01:30,760 asking you to share your career highlights first or 30 00:01:30,760 --> 00:01:32,980 any turning points that you think we all can learn 31 00:01:32,980 --> 00:01:35,020 from you. Yeah, sure. So 32 00:01:35,020 --> 00:01:38,580 I always start with the terrible joke of I wasn't 33 00:01:38,580 --> 00:01:40,480 supposed to be a software tester, I was supposed to 34 00:01:40,480 --> 00:01:43,340 be a rock star, but things kind of went a bit 35 00:01:43,340 --> 00:01:46,860 different trajectory. So yeah, I actually studied 36 00:01:46,860 --> 00:01:50,820 music at university, music and technology. I 37 00:01:50,820 --> 00:01:53,340 was always interested in computers, even as a kid, 38 00:01:53,520 --> 00:01:56,040 but I was also very interested in music as 39 00:01:56,040 --> 00:01:58,200 well. But my first job was actually 40 00:01:58,200 --> 00:02:03,140 testing music notation. So I managed to carve myself 41 00:02:03,140 --> 00:02:06,420 a little bit of a niche of having the ability to 42 00:02:06,420 --> 00:02:09,000 work with technology but also being able to read 43 00:02:09,000 --> 00:02:11,600 music to an advanced degree that meant that I could 44 00:02:11,600 --> 00:02:14,080 sort of get into the testing role. And 45 00:02:14,080 --> 00:02:16,460 it was more of a I wanted to get myself a foot in 46 00:02:16,460 --> 00:02:19,600 the door into tech because I wanted to write music 47 00:02:19,600 --> 00:02:22,120 for video games. And I'd heard the way through 48 00:02:22,120 --> 00:02:25,060 into the games industry was through testing, I couldn't 49 00:02:25,060 --> 00:02:27,280 get a testing job and sort of that sort of classic 50 00:02:27,280 --> 00:02:29,840 thing of we're looking for a junior, must have three 51 00:02:29,840 --> 00:02:32,100 years experience, that sort of thing. But 52 00:02:32,100 --> 00:02:35,000 yeah, so I got this job as a tester and actually 53 00:02:35,000 --> 00:02:38,800 found that I really enjoyed the process of testing. And 54 00:02:38,800 --> 00:02:42,120 then I was quite lucky early on that I had a 55 00:02:42,120 --> 00:02:45,960 mentor who saw my interest in technology and coached 56 00:02:45,960 --> 00:02:48,180 me into getting into test automation quite early on. 57 00:02:48,780 --> 00:02:51,100 So like most of my career has 58 00:02:51,100 --> 00:02:53,540 been around the test automation space, working 59 00:02:53,540 --> 00:02:57,500 across a lot of different companies or startups, big 60 00:02:57,500 --> 00:03:01,100 enterprise places. I moved into contracting about 61 00:03:01,100 --> 00:03:04,300 five, six years into my career and that gave me an 62 00:03:04,300 --> 00:03:06,320 opportunity to move around. And 63 00:03:06,320 --> 00:03:08,740 it was sort of about six years into that that I 64 00:03:08,740 --> 00:03:11,680 started teaching as well, start teaching testing. So 65 00:03:11,680 --> 00:03:15,560 talking about testing web APIs, my first workshop 66 00:03:15,560 --> 00:03:19,460 that I built was around teaching testers how to use 67 00:03:19,460 --> 00:03:23,580 APIs, how to test them, how to apply sort of 68 00:03:23,580 --> 00:03:26,440 exploratory testing and heuristics and stuff, but 69 00:03:26,448 --> 00:03:28,320 as well as automation. Found 70 00:03:28,320 --> 00:03:30,480 that I got really real good taste of that and 71 00:03:30,480 --> 00:03:33,640 that rock style mentality of touring the world and 72 00:03:33,640 --> 00:03:36,600 going to conferences and teaching and talking about, 73 00:03:36,860 --> 00:03:39,840 not just web APIs, but also about testing and 74 00:03:39,840 --> 00:03:42,240 automation in general, as well. I 75 00:03:42,248 --> 00:03:45,140 then teamed up with a chap called Richard Bradshaw 76 00:03:45,140 --> 00:03:48,100 who's a friendly tester and we started running a course 77 00:03:48,100 --> 00:03:49,640 called Automation Testing. So 78 00:03:49,640 --> 00:03:53,620 whilst I was doing testing, I was also teaching and 79 00:03:53,620 --> 00:03:56,640 learning a lot through various conferences and 80 00:03:56,640 --> 00:04:00,560 things I have had. And yeah, sort of about sort 81 00:04:00,560 --> 00:04:03,280 of 10 years in, that's when I started writing my 82 00:04:03,280 --> 00:04:06,000 first book, Testing Web APIs. And 83 00:04:06,000 --> 00:04:08,180 then yeah, sort of bring up to now on quality 84 00:04:08,180 --> 00:04:11,080 engineer for John Lewis partnership. I'm 85 00:04:11,080 --> 00:04:14,480 sort of still working in the test automation space, 86 00:04:14,480 --> 00:04:17,660 but I'm sort of thinking about quality in general 87 00:04:17,660 --> 00:04:21,300 these days, not just about the quality of the product, 88 00:04:21,360 --> 00:04:23,340 but the quality of the work, the quality of the 89 00:04:23,340 --> 00:04:28,460 testing and supporting my team so that they can be 90 00:04:28,460 --> 00:04:30,200 the best that they can be. Thanks 91 00:04:30,200 --> 00:04:32,120 for sharing your story. I think it's pretty 92 00:04:32,120 --> 00:04:34,400 interesting testing music notation. I 93 00:04:34,400 --> 00:04:38,000 haven't heard it before. So I think that's how you got 94 00:04:38,000 --> 00:04:39,940 into this whole testing journey. I 95 00:04:39,940 --> 00:04:41,600 think that's pretty interesting to hear. Thanks 96 00:04:41,600 --> 00:04:44,540 for sharing that. Hey, thank you for being part 97 00:04:44,540 --> 00:04:46,860 of the TechLyJunior community. This 98 00:04:46,860 --> 00:04:49,380 show wouldn't be the same without your ears. And 99 00:04:49,380 --> 00:04:52,420 you are the reason this show exists. If 100 00:04:52,420 --> 00:04:55,040 you're loving TLJ and want to see it keep on growing, 101 00:04:55,620 --> 00:04:59,600 consider becoming a patron at TechLyJunior .dev .patreon 102 00:04:59,600 --> 00:05:03,420 or buying me a coffee at TechLyJunior .dev .coffee. 103 00:05:04,140 --> 00:05:06,140 Every little bit helps fill the 104 00:05:06,140 --> 00:05:08,960 research, editing and sleepless nights that go 105 00:05:08,960 --> 00:05:12,240 into making this show the best it can be. Thanks 106 00:05:12,240 --> 00:05:14,560 for being the best listeners any podcast could 107 00:05:14,560 --> 00:05:16,880 ask for. And now let's get back to our 108 00:05:16,880 --> 00:05:19,340 episode. So you mentioned you have been 109 00:05:19,340 --> 00:05:22,780 doing a lot of automation, API automation, web API 110 00:05:22,780 --> 00:05:25,880 automation, maybe a little bit of background. How 111 00:05:25,880 --> 00:05:27,920 did you come up with the idea to write this book? 112 00:05:28,100 --> 00:05:29,060 What kind of problems that 113 00:05:29,060 --> 00:05:31,360 you're trying to solve? Well, 114 00:05:31,600 --> 00:05:35,360 as I said, it started with the workshop initially. I 115 00:05:35,360 --> 00:05:37,880 was keen to just sort of put something out there and 116 00:05:37,880 --> 00:05:40,140 do a bit of public speaking. So 117 00:05:40,140 --> 00:05:43,760 I kind of set myself the challenge of building this 118 00:05:43,760 --> 00:05:46,260 workshop. And then I went out with a 119 00:05:46,260 --> 00:05:49,200 guy I used to run London Tester Gathering. His 120 00:05:49,200 --> 00:05:51,380 name is Tony Bruce. And he used to run these 121 00:05:51,380 --> 00:05:53,520 collection of workshops. And we were having a drink and 122 00:05:53,520 --> 00:05:55,960 I was like, I've got this idea of an API testing 123 00:05:55,960 --> 00:05:57,300 workshop. And he was like, right, you're 124 00:05:57,300 --> 00:05:58,260 in. You're doing it. And 125 00:05:58,260 --> 00:06:00,600 I was like, oh, OK. So it kind of just sort of 126 00:06:00,600 --> 00:06:04,200 kicked off like that. But something that's always kind 127 00:06:04,200 --> 00:06:08,520 of threaded in a lot of my teaching is that I like 128 00:06:08,520 --> 00:06:13,440 to teach practical skills, but also have a foundation 129 00:06:13,440 --> 00:06:19,180 of mindset and philosophy towards how we're doing our 130 00:06:19,180 --> 00:06:21,420 testing. So I'm a huge advocate for 131 00:06:21,420 --> 00:06:23,500 exploratory testing. So I love doing automation, 132 00:06:23,700 --> 00:06:25,600 but I think exploratory testing is a massively 133 00:06:25,600 --> 00:06:28,560 important skill for all testers or for anyone who's 134 00:06:28,560 --> 00:06:31,140 involved in kind of quality space. But 135 00:06:31,147 --> 00:06:34,340 teaching that is quite tricky. Whereas 136 00:06:34,340 --> 00:06:38,140 if I can teach someone to test a web API in an 137 00:06:38,140 --> 00:06:40,440 exploratory testing context, then they kind of 138 00:06:40,440 --> 00:06:43,040 implicitly learn that sort of information. So 139 00:06:43,040 --> 00:06:45,520 that was kind of what I was doing with the book, 140 00:06:45,680 --> 00:06:48,560 with testing web APIs. I developed this body of 141 00:06:48,560 --> 00:06:51,860 work, this understanding of how I approach testing web 142 00:06:51,860 --> 00:06:54,780 APIs. And at a couple of the temps 143 00:06:54,780 --> 00:06:57,540 at trying to write it, people kept sort of 144 00:06:57,540 --> 00:06:59,760 encouraging me, but I was sort of like, there's not 145 00:06:59,760 --> 00:07:02,720 enough material here. Turns out I was wrong. But 146 00:07:02,720 --> 00:07:06,880 I ended up watching some online gaming streamer who 147 00:07:06,880 --> 00:07:09,220 was also an author. And someone asked him, like, 148 00:07:09,240 --> 00:07:10,200 how do you write a book? And 149 00:07:10,200 --> 00:07:13,540 he was like, one page a day, or a page at a time 150 00:07:13,540 --> 00:07:15,800 sort of thing. So I was like, oh, OK, I'll 151 00:07:15,800 --> 00:07:18,520 give that a try. So I literally wrote a page a 152 00:07:18,520 --> 00:07:21,340 day for a month. And before I knew it, I had 153 00:07:21,340 --> 00:07:23,580 my first chapter. I thought, actually, I think 154 00:07:23,580 --> 00:07:25,940 I could do this. I'd been blogging a lot 155 00:07:25,940 --> 00:07:29,340 previously and writing articles and stuff for 156 00:07:29,340 --> 00:07:30,980 companies and teaching materials. So 157 00:07:30,980 --> 00:07:33,080 I'd sort of built up that skill set. So 158 00:07:33,080 --> 00:07:35,620 it was kind of, yeah, it was sort of this sort of 159 00:07:35,620 --> 00:07:38,600 the training that I was doing on the side, the 160 00:07:38,600 --> 00:07:40,580 growing interest in writing. And 161 00:07:40,580 --> 00:07:43,660 then, like, yeah, I say, this sort of small gradual 162 00:07:43,660 --> 00:07:46,100 process of building something up over time to 163 00:07:46,100 --> 00:07:48,440 end up with a body of work. Right. 164 00:07:48,860 --> 00:07:50,640 And when I read your book, actually, it's pretty 165 00:07:50,640 --> 00:07:52,420 interesting. I was expecting actually to 166 00:07:52,420 --> 00:07:55,260 read a lot of materials about automation, how you 167 00:07:55,260 --> 00:07:57,580 deal with, I don't know, postman, rest APIs and 168 00:07:57,580 --> 00:07:59,580 things like that. But actually, the way you 169 00:07:59,580 --> 00:08:02,260 explain stuff in your book, it's a mix of mindset, 170 00:08:02,480 --> 00:08:04,920 philosophy, and also different strategies, right? 171 00:08:05,080 --> 00:08:06,320 Which I think in your book, you 172 00:08:06,320 --> 00:08:08,740 mentioned it, like the holistic approach. So 173 00:08:08,740 --> 00:08:11,460 tell us why it is very important for testing 174 00:08:11,460 --> 00:08:14,960 strategy or testing approach or methodology. Has 175 00:08:14,960 --> 00:08:17,100 to be more holistic. And what do you mean by 176 00:08:17,100 --> 00:08:19,880 holistic here? So when we take a step back, 177 00:08:19,920 --> 00:08:21,620 when we're talking about holistic, we're talking 178 00:08:21,620 --> 00:08:26,180 about basically different activities in the context 179 00:08:26,180 --> 00:08:29,060 of the testing strategy, different activities that 180 00:08:29,060 --> 00:08:33,340 are being executed to address different types of 181 00:08:33,340 --> 00:08:36,020 risks. And that's why I think holistic 182 00:08:36,020 --> 00:08:39,760 strategies are necessary, because we have to handle 183 00:08:39,760 --> 00:08:42,240 lots of different types of risks that can impact our 184 00:08:42,240 --> 00:08:45,420 product. So for example, things like 185 00:08:45,420 --> 00:08:49,860 automation is useful, but it is very much targeted 186 00:08:49,860 --> 00:08:52,220 at a specific set of risks, which tend to be 187 00:08:52,220 --> 00:08:55,220 the sort of kind of the functional, the correctness 188 00:08:55,220 --> 00:08:58,640 of a product. They are change detectors. So 189 00:08:58,640 --> 00:09:02,320 they are there to help us determine whether or not 190 00:09:02,320 --> 00:09:04,880 the system has changed intentionally or 191 00:09:04,880 --> 00:09:07,260 unintentionally. But that doesn't help you with 192 00:09:07,260 --> 00:09:08,660 things like performance. That 193 00:09:08,660 --> 00:09:11,940 doesn't help you with issues around implementing 194 00:09:11,940 --> 00:09:16,120 the wrong thing in the first place, or how those 195 00:09:16,120 --> 00:09:19,960 fringe or edge cases occur in our APIs. All 196 00:09:19,960 --> 00:09:22,880 those different types of risks, security, how the 197 00:09:22,880 --> 00:09:25,660 end users can use it, how it interacts with other 198 00:09:25,660 --> 00:09:27,600 things. All these are different types 199 00:09:27,600 --> 00:09:30,360 of distinct risks that we may or may not care about. 200 00:09:30,660 --> 00:09:32,300 So quality comes into this. 201 00:09:32,500 --> 00:09:34,040 Like what does quality mean to 202 00:09:34,040 --> 00:09:36,800 our end users and what risks could impact that? 203 00:09:37,180 --> 00:09:38,580 That determines what type of 204 00:09:38,580 --> 00:09:42,080 activities we do. The challenge is that if we go 205 00:09:42,080 --> 00:09:44,480 in with what's the normal mindset towards testing, 206 00:09:44,560 --> 00:09:47,460 which is running test scripts, having them 207 00:09:47,460 --> 00:09:50,040 executed manually or automated, is you get this 208 00:09:50,040 --> 00:09:52,220 sort of idea of a monoculture. We're 209 00:09:52,220 --> 00:09:55,080 only focused on a very specific type of risk, and 210 00:09:55,080 --> 00:09:58,220 we're ignoring these other aspects to our potential 211 00:09:58,220 --> 00:10:01,400 detriment or to a negative impact. So 212 00:10:01,400 --> 00:10:04,400 there's that aspect. The other side as well is 213 00:10:04,400 --> 00:10:07,180 that, say, every context is different. So 214 00:10:07,180 --> 00:10:11,000 if I'm testing APIs for... So 215 00:10:11,000 --> 00:10:13,540 I worked for HMRC for a while, and I was on their 216 00:10:13,540 --> 00:10:17,960 tax platform. Performance is important. Accuracy 217 00:10:17,960 --> 00:10:20,980 of calculations is important. Whereas 218 00:10:20,980 --> 00:10:24,380 perhaps a classic one that people were talking about 219 00:10:24,380 --> 00:10:27,000 back in the day was like Pokemon Go. People 220 00:10:27,000 --> 00:10:30,460 probably care much more about performance, and they 221 00:10:30,460 --> 00:10:33,620 may care more about security, like personal data 222 00:10:33,620 --> 00:10:36,360 being shared across, or people stealing all of your 223 00:10:36,360 --> 00:10:39,320 inventory from your account or something like that. So 224 00:10:39,320 --> 00:10:42,560 different contexts have different needs, different 225 00:10:42,560 --> 00:10:45,740 aspects of quality. So again, holistic being 226 00:10:45,740 --> 00:10:49,220 holistic means you're being responsive to what's going 227 00:10:49,220 --> 00:10:51,700 on around you, what is that you're dealing with, 228 00:10:51,920 --> 00:10:55,920 you're tuning into that, rather than saying one size 229 00:10:55,920 --> 00:11:00,700 fits all, trying to square peg it around whole. That 230 00:11:00,700 --> 00:11:03,540 sort of idea. Yeah. So thanks for mentioning 231 00:11:03,540 --> 00:11:06,380 this understanding about context and risk. So 232 00:11:06,380 --> 00:11:08,580 I think that's the main theme all over your book. 233 00:11:08,780 --> 00:11:11,160 So you cover why specifically 234 00:11:11,160 --> 00:11:13,340 we need to care about certain stuff, and what 235 00:11:13,340 --> 00:11:16,120 kind of risks are we trying to solve here. And 236 00:11:16,120 --> 00:11:19,220 you also emphasize that we as testers or developers, 237 00:11:19,560 --> 00:11:22,320 we have to understand the problem first before we 238 00:11:22,320 --> 00:11:24,820 actually approach testing overall, rather than 239 00:11:24,820 --> 00:11:27,560 starting to just write test scripts and things like 240 00:11:27,560 --> 00:11:29,980 that. So maybe can you explain how 241 00:11:29,980 --> 00:11:32,000 can we actually start by understanding the problem 242 00:11:32,000 --> 00:11:34,960 better, and why is it important? So 243 00:11:34,960 --> 00:11:38,360 I'm always a big advocate of if you're starting a 244 00:11:38,360 --> 00:11:41,940 new project, is just generally asking questions 245 00:11:41,940 --> 00:11:45,700 and exploring, but not necessarily exploring in a 246 00:11:45,700 --> 00:11:47,860 way to make judgments. You'll 247 00:11:47,860 --> 00:11:50,820 do that at a later point, but it's more of a sort 248 00:11:50,820 --> 00:11:54,180 of exploring the product, exploring the people who 249 00:11:54,180 --> 00:11:56,960 work on the product. That's why I quite like the 10 250 00:11:56,960 --> 00:11:59,480 Ps of Testability by Robmini and Ashwinta, 251 00:11:59,680 --> 00:12:02,340 because it breaks down a context into these distinct 252 00:12:02,340 --> 00:12:04,320 areas. So looking at the people, 253 00:12:04,580 --> 00:12:07,880 looking at the process, how is the product built? What 254 00:12:07,880 --> 00:12:09,960 technologies are we using? How 255 00:12:09,960 --> 00:12:12,460 do we get it from our computers in front of other 256 00:12:12,460 --> 00:12:14,620 people? It's like the pipeline process. 257 00:12:15,420 --> 00:12:16,940 So understanding all of that 258 00:12:16,940 --> 00:12:20,960 information helps us to, again, better appreciate 259 00:12:21,640 --> 00:12:25,460 what challenges we face as a team. It 260 00:12:25,460 --> 00:12:29,200 helps us understand who our own users are, and what 261 00:12:29,200 --> 00:12:31,580 are we trying to achieve for them. And 262 00:12:31,580 --> 00:12:33,640 it's by putting all of this sort of information 263 00:12:33,640 --> 00:12:36,620 together that we start to identify those opportunities. 264 00:12:37,260 --> 00:12:39,840 So for me, testing is always 265 00:12:39,840 --> 00:12:43,340 about supporting a team, making sure that they're 266 00:12:43,340 --> 00:12:46,580 informed and they're making right decisions, they're 267 00:12:46,580 --> 00:12:48,660 making the most valuable decisions at the best of 268 00:12:48,660 --> 00:12:51,140 times. So yeah, having that sort of 269 00:12:51,140 --> 00:12:54,340 context in place, it becomes easier to identify 270 00:12:54,340 --> 00:12:57,900 those opportunities to sort of elevate the team so 271 00:12:57,900 --> 00:13:00,800 that they can build a higher quality product. So 272 00:13:00,800 --> 00:13:01,800 that's a big thing as well. And 273 00:13:01,800 --> 00:13:04,580 I was like, testing doesn't necessarily, like I could 274 00:13:04,580 --> 00:13:06,120 do all the testing in the world, but we could still 275 00:13:06,120 --> 00:13:08,360 end up with a very low quality product. It 276 00:13:08,360 --> 00:13:12,340 may work, but people may hate it, or it may work 277 00:13:12,340 --> 00:13:14,680 in a certain way, but as soon as somebody presses 278 00:13:14,680 --> 00:13:17,460 the shiny red button on page three, the whole thing 279 00:13:17,460 --> 00:13:19,780 falls over. So yeah, so gathering all of 280 00:13:19,780 --> 00:13:22,080 that kind of information helps us identify those 281 00:13:22,080 --> 00:13:24,800 opportunities. And then from there, we can 282 00:13:24,800 --> 00:13:27,740 start being strategic about which opportunities are we 283 00:13:27,740 --> 00:13:30,480 going to follow, how are we going to measure those 284 00:13:30,480 --> 00:13:34,140 opportunities and those ways in which we address those 285 00:13:34,140 --> 00:13:36,960 opportunities are valuable and we're not going off 286 00:13:36,960 --> 00:13:39,540 track. Yeah, but it all comes from that 287 00:13:39,540 --> 00:13:42,040 sort of kind of information gathering process at the 288 00:13:42,040 --> 00:13:44,600 start. Yeah, so I think another thing 289 00:13:44,600 --> 00:13:47,020 you mentioned in the book after you asked questions 290 00:13:47,020 --> 00:13:48,360 and things like that, right? It's 291 00:13:48,360 --> 00:13:50,840 very important to build shared understanding among 292 00:13:50,840 --> 00:13:53,840 different team members, maybe in the teams, right? 293 00:13:54,260 --> 00:13:55,940 Because I think one common practice 294 00:13:55,940 --> 00:13:58,980 I see about testing, a lot of teams actually kind of 295 00:13:58,980 --> 00:14:00,620 like throw it over the wall, you know, like they 296 00:14:00,620 --> 00:14:02,820 have a team of testers, you know, you pass them 297 00:14:02,820 --> 00:14:05,120 the requirements, pass them the binaries, right, and 298 00:14:05,120 --> 00:14:07,180 let them test. Maybe it's more like a black box 299 00:14:07,180 --> 00:14:09,380 approach, but at the same time, probably the shared 300 00:14:09,380 --> 00:14:11,700 understanding is not there, right? And 301 00:14:11,700 --> 00:14:14,540 I think one thing that I really like in the book is 302 00:14:14,540 --> 00:14:17,960 about this Venn diagram, and you kind of like bring 303 00:14:17,960 --> 00:14:20,600 that all over different chapters, right, when you 304 00:14:20,600 --> 00:14:23,160 cover different strategies, maybe elaborate a little 305 00:14:23,160 --> 00:14:25,520 bit more about this Venn diagram, how can we 306 00:14:25,520 --> 00:14:28,700 actually use it in our testing strategy? Yeah, 307 00:14:28,860 --> 00:14:32,420 so it's based off a model, a visual model, that James 308 00:14:32,420 --> 00:14:36,580 Lindsay created to kind of explain the value of 309 00:14:36,580 --> 00:14:39,320 exploratory testing, but I remember when he showed it 310 00:14:39,320 --> 00:14:41,260 to me and I really liked it, but I actually thought 311 00:14:41,260 --> 00:14:44,140 that it would be applied to testing as a whole. So 312 00:14:44,140 --> 00:14:46,420 the idea is, like you say, it's a Venn diagram. We 313 00:14:46,420 --> 00:14:50,060 have one circle, which is the imagination, and another 314 00:14:50,060 --> 00:14:52,480 circle, which is implementation. So 315 00:14:52,480 --> 00:14:55,160 on the imagination side, this is where we are 316 00:14:55,160 --> 00:14:58,780 testing to learn about what it is that we want in our 317 00:14:58,780 --> 00:15:02,080 product, what do we want to build, and inside that, 318 00:15:02,100 --> 00:15:04,920 we will have explicit information, so like say 319 00:15:04,920 --> 00:15:08,480 like requirements, acceptance criteria, test cases, 320 00:15:09,040 --> 00:15:12,140 documentation, but then we also have implicit and 321 00:15:12,140 --> 00:15:14,080 tacit information there as well. So 322 00:15:14,080 --> 00:15:16,840 why are we building this product? When 323 00:15:16,840 --> 00:15:19,360 someone says relevant results, what do you mean 324 00:15:19,360 --> 00:15:22,180 by relevant? Relevant to who? That's 325 00:15:22,180 --> 00:15:24,320 where the sort of misunderstandings come from. 326 00:15:24,740 --> 00:15:26,840 So we want to ask questions 327 00:15:26,840 --> 00:15:29,160 there to dispel those incorrect assumptions, 328 00:15:30,140 --> 00:15:32,360 misunderstandings across the team. And 329 00:15:32,360 --> 00:15:36,060 the idea is, the more we learn about that side, we 330 00:15:36,060 --> 00:15:38,680 apply the same thing on the implementation side. So 331 00:15:38,680 --> 00:15:41,000 the implementation side is the product, the thing that 332 00:15:41,000 --> 00:15:44,360 exists. Again, if we are only testing 333 00:15:44,360 --> 00:15:46,820 based on explicit information, so again, 334 00:15:47,220 --> 00:15:49,760 acceptance criteria, test scripts, requirements, that 335 00:15:49,760 --> 00:15:52,360 sort of stuff, we're only testing a small portion of 336 00:15:52,360 --> 00:15:54,460 actually how the product behaves. So 337 00:15:54,468 --> 00:15:57,260 things like exploratory testing and monitoring can 338 00:15:57,260 --> 00:15:59,860 be really useful because those sort of activities 339 00:15:59,860 --> 00:16:03,340 help us learn more about how the product actually 340 00:16:03,340 --> 00:16:07,420 really behaves. So by learning more about how 341 00:16:07,420 --> 00:16:09,760 the product actually behaves, and by learning 342 00:16:09,760 --> 00:16:12,600 more about how we want the product to work in the 343 00:16:12,600 --> 00:16:15,740 first place, the more we can overlap these two areas 344 00:16:15,740 --> 00:16:20,100 so that we can make better informed decisions. Like 345 00:16:20,100 --> 00:16:22,640 I said, if we go for just a monoculture approach, 346 00:16:22,640 --> 00:16:25,480 we're just running test scripts, then you get a 347 00:16:25,480 --> 00:16:27,960 little bit of an overlap in that FEND diagram 348 00:16:27,960 --> 00:16:31,100 because you are kind of using your explicit 349 00:16:31,100 --> 00:16:34,020 understanding of what using the product is used to 350 00:16:34,020 --> 00:16:37,480 test the product, and you learn some information, but 351 00:16:37,480 --> 00:16:39,800 you are missing out on so much more. So 352 00:16:39,800 --> 00:16:42,180 that's why I like using that model, because it 353 00:16:42,180 --> 00:16:45,120 communicates for me the goal of testing, which is 354 00:16:45,120 --> 00:16:48,560 to find out as much about both of these items and 355 00:16:48,560 --> 00:16:51,320 get that overlap as much as possible. It'll 356 00:16:51,320 --> 00:16:54,780 never be 100%, but you're always striving towards it. 357 00:16:55,120 --> 00:16:56,380 But then also as well, again, 358 00:16:56,460 --> 00:17:00,440 it goes back to this risk aspect of some risks live 359 00:17:00,440 --> 00:17:03,920 more in the implementation side, some live more in 360 00:17:03,920 --> 00:17:07,060 the imagination side, and then some exist in that 361 00:17:07,060 --> 00:17:09,920 sort of overlap because as our product gets more 362 00:17:09,920 --> 00:17:12,540 complex, things like regression become an 363 00:17:12,540 --> 00:17:15,240 important factor, but they're not the be all and 364 00:17:15,240 --> 00:17:17,300 end all. Again, it's that spread of all of 365 00:17:17,300 --> 00:17:19,260 them. So yeah, that's why I'm sort 366 00:17:19,260 --> 00:17:21,099 of a big fan of that model. And 367 00:17:21,099 --> 00:17:23,300 as you say, it's a great teaching aid in the book 368 00:17:23,300 --> 00:17:25,900 because I can move across the model in different 369 00:17:25,900 --> 00:17:27,980 places and talk about how these different activities 370 00:17:27,980 --> 00:17:30,400 work in different areas. Yeah, 371 00:17:30,600 --> 00:17:33,060 and I think not to mention also for different areas, 372 00:17:33,200 --> 00:17:36,160 like the imagination will have some testing strategies 373 00:17:36,160 --> 00:17:38,800 that kind of like more focus towards that, things 374 00:17:38,800 --> 00:17:41,820 like for example, contract testing, right, and also 375 00:17:41,820 --> 00:17:45,240 test API designs, and also on the implementation side, 376 00:17:45,240 --> 00:17:46,820 maybe there are other aspects as well. So 377 00:17:46,820 --> 00:17:48,940 you kind of like bring the whole holistic approach to 378 00:17:48,940 --> 00:17:52,840 testing, and it's not just like an automation in terms 379 00:17:52,840 --> 00:17:55,160 of API step by step, right? So 380 00:17:55,160 --> 00:17:57,600 I think it's very interesting definitely. And 381 00:17:57,600 --> 00:18:00,480 in the overlap, right, so if we can make it larger 382 00:18:00,480 --> 00:18:02,320 and larger, we kind of like align both the 383 00:18:02,320 --> 00:18:03,940 imagination and implementation. And 384 00:18:03,940 --> 00:18:06,980 I think that's where the testing strategy, where you 385 00:18:06,980 --> 00:18:09,780 build a lot more automation to cover both areas, I 386 00:18:09,780 --> 00:18:11,620 think will be more powerful. So 387 00:18:11,620 --> 00:18:13,680 I think throughout the book, you will see a lot 388 00:18:13,680 --> 00:18:15,760 of these Venn diagrams. So for people who are 389 00:18:15,760 --> 00:18:17,720 interested, you can check out the book as well. I 390 00:18:17,720 --> 00:18:21,100 think it's really powerful framework, frame along all 391 00:18:21,100 --> 00:18:24,120 these testing strategy. So after we understand about 392 00:18:24,120 --> 00:18:26,660 the importance of this testing approach and 393 00:18:26,660 --> 00:18:28,960 understanding context and risk, right, we need to 394 00:18:28,960 --> 00:18:31,860 pick the testing strategies. And 395 00:18:31,860 --> 00:18:33,860 I think you emphasize a lot in this conversation 396 00:18:33,860 --> 00:18:36,360 already as well about the risk. I 397 00:18:36,360 --> 00:18:40,340 think sometimes this is not intuitive for many teams, I 398 00:18:40,340 --> 00:18:42,980 think, because when we talk about testing, right, they 399 00:18:42,980 --> 00:18:45,040 always come up with, okay, what are the requirements, 400 00:18:45,360 --> 00:18:48,960 right, what users need to do, but they actually don't 401 00:18:48,960 --> 00:18:51,140 talk it in the perspective of risk. So 402 00:18:51,140 --> 00:18:53,200 maybe if you can explain a little bit more, how can 403 00:18:53,200 --> 00:18:55,720 we actually start building our test strategy using 404 00:18:55,720 --> 00:19:00,120 this risk perspective? Yeah, I think it's as much, like 405 00:19:00,120 --> 00:19:03,260 you say, it's about mindset as it is in terms of 406 00:19:03,260 --> 00:19:05,800 specific techniques and approaches. We're 407 00:19:05,800 --> 00:19:09,480 always under so much pressure to deliver. And 408 00:19:09,480 --> 00:19:12,160 I think because of that, we tend to sort of focus 409 00:19:12,160 --> 00:19:15,720 on the output at the end, the artifact. So 410 00:19:15,720 --> 00:19:17,520 when we talk about the context of testing, we're 411 00:19:17,520 --> 00:19:20,560 talking about the tests that were done. So 412 00:19:20,560 --> 00:19:23,520 I think it's like when we're trying to get people 413 00:19:23,520 --> 00:19:27,300 to do sort of risk based testing, the first thing is 414 00:19:27,300 --> 00:19:31,440 really to get this, it's funny, I've been seeing 415 00:19:31,440 --> 00:19:34,700 this quite a lot lately again, and it does annoy 416 00:19:34,700 --> 00:19:37,860 me as it's talking about testing types, types of 417 00:19:37,860 --> 00:19:40,900 testing. So people build strategies or 418 00:19:40,900 --> 00:19:43,300 approaches to testing around testing types. So 419 00:19:43,300 --> 00:19:45,420 you have to do integration testing, you have to do 420 00:19:45,420 --> 00:19:48,620 functional testing. It's like, yes, those are types 421 00:19:48,620 --> 00:19:51,560 of testing, but that's, you know, it's trying to think 422 00:19:51,560 --> 00:19:54,240 of a good analogy. It's like talking about types of 423 00:19:54,240 --> 00:19:57,060 tools, like, oh, you know, we're going to do some 424 00:19:57,060 --> 00:19:58,320 DIY. Well, I have to have a 425 00:19:58,320 --> 00:20:00,220 screwdriver. I have to have a hammer. So 426 00:20:00,220 --> 00:20:02,420 it's fine, but you're painting a wall. So 427 00:20:02,420 --> 00:20:04,460 those things aren't going to be very useful for you. 428 00:20:04,460 --> 00:20:07,440 So I think, yeah, it's that 429 00:20:07,440 --> 00:20:10,660 mindset of trying not to think of it as types or 430 00:20:10,660 --> 00:20:12,960 boxes of testing. Think about the product, think 431 00:20:12,960 --> 00:20:15,860 about the end goal and what risks impact that. 432 00:20:16,300 --> 00:20:18,000 Then in terms of like the next 433 00:20:18,000 --> 00:20:20,960 sort of tricky part, he's getting people to be sort 434 00:20:20,960 --> 00:20:24,400 of kind of risk based and risk analysis. So 435 00:20:24,400 --> 00:20:27,040 this is where like a tester or anyone who's 436 00:20:27,040 --> 00:20:29,640 interested in qualities, like number one tool is 437 00:20:29,640 --> 00:20:32,340 questions. He's asking questions and going, 438 00:20:32,540 --> 00:20:36,420 you know, what will happen if X, Y and Z happens? Or 439 00:20:36,420 --> 00:20:38,000 what would be the result of this? What 440 00:20:38,000 --> 00:20:41,640 do you mean by that? So using like five Ws and an 441 00:20:41,640 --> 00:20:45,320 H, Y, Y, who, when and how. Yet 442 00:20:45,320 --> 00:20:48,700 those sort of primers to ask questions is a great 443 00:20:48,700 --> 00:20:51,820 place to start. Then you've got frameworks like 444 00:20:51,820 --> 00:20:56,520 risk storming by Barry Vandal, who has built this 445 00:20:56,520 --> 00:20:59,060 like brilliant online asynchronous tool that you 446 00:20:59,060 --> 00:21:01,140 can use. He's literally risk storming 447 00:21:01,140 --> 00:21:04,080 online. Just Google that and I can't 448 00:21:04,080 --> 00:21:06,380 URL. Sorry, Barry. But 449 00:21:06,380 --> 00:21:09,140 using sort of kind of more sort of heuristic based 450 00:21:09,140 --> 00:21:11,120 techniques. So Elizabeth Hendrickson, I 451 00:21:11,120 --> 00:21:14,080 think, came up with the idea of the newspaper game. 452 00:21:14,640 --> 00:21:17,020 So imagine a newspaper article, 453 00:21:17,520 --> 00:21:20,260 like a headline in the paper and use that as a 454 00:21:20,260 --> 00:21:24,020 trigger to what would cause company X leaks customer 455 00:21:24,020 --> 00:21:26,660 data, right? Well, what happened to cause that 456 00:21:26,660 --> 00:21:29,020 headline? And you sort of follow the 457 00:21:29,020 --> 00:21:31,680 story there. There are frameworks that we can 458 00:21:31,680 --> 00:21:33,960 use that we can follow. So like risk storming is 459 00:21:33,960 --> 00:21:36,140 really good because it helps you identify quality, 460 00:21:36,460 --> 00:21:40,180 then it helps you identify risks, and then it helps 461 00:21:40,180 --> 00:21:42,920 you identify what's testing you want to do for those 462 00:21:42,920 --> 00:21:45,420 risks. Whereas, you know, that's much more 463 00:21:45,420 --> 00:21:48,160 structured workshop based thing. Whereas, 464 00:21:48,320 --> 00:21:50,540 yeah, like the headline games, things like test a 465 00:21:50,540 --> 00:21:52,540 bleep on my talks as well. Those 466 00:21:52,540 --> 00:21:54,960 are sort of much more informal things like five 467 00:21:54,960 --> 00:21:57,540 Ws and H. It's just a primary trigger to 468 00:21:57,540 --> 00:21:59,740 keep to ask those questions. Yeah. 469 00:22:00,120 --> 00:22:02,740 So I think if people adopt this kind of a risk based 470 00:22:02,740 --> 00:22:06,140 approach, I think it will be much more valuable in 471 00:22:06,140 --> 00:22:08,640 terms of the output of your testing activities, 472 00:22:08,920 --> 00:22:10,620 right? Not just producing, you know, 473 00:22:10,680 --> 00:22:13,380 hundreds of test scripts, just UI automation, for 474 00:22:13,380 --> 00:22:16,260 example, or just user based approach. But 475 00:22:16,260 --> 00:22:18,680 you kind of like miss the whole aspects of different 476 00:22:18,680 --> 00:22:20,880 risks, right? Because we cannot test everything. 477 00:22:20,980 --> 00:22:22,620 Definitely, there will be too many. 478 00:22:22,620 --> 00:22:24,140 But yeah, how should we 479 00:22:24,140 --> 00:22:26,240 prioritize what kind of things that we should test? 480 00:22:26,680 --> 00:22:27,540 I think this risk based 481 00:22:27,540 --> 00:22:29,820 approach is really important. And 482 00:22:29,820 --> 00:22:32,420 before we actually come up with those prioritized 483 00:22:32,420 --> 00:22:34,940 risks, right, you also mentioned that we have to 484 00:22:34,940 --> 00:22:38,040 know about the prioritized quality attributes. But 485 00:22:38,040 --> 00:22:40,180 let's say, but we talk about quality first, right? 486 00:22:40,200 --> 00:22:41,620 Because I think the whole purpose 487 00:22:41,620 --> 00:22:44,640 of testing normally is to a certain standard or 488 00:22:44,640 --> 00:22:46,920 quality of the product. So what do you mean by 489 00:22:46,920 --> 00:22:49,160 quality? And how do we actually think 490 00:22:49,160 --> 00:22:51,460 about the quality attributes? Yeah, 491 00:22:51,480 --> 00:22:53,780 so I think that's the crux of it. Of 492 00:22:53,780 --> 00:22:56,420 all of it is that, yeah, like what does quality 493 00:22:56,420 --> 00:22:59,700 mean? So whenever I'm sort of like 494 00:22:59,700 --> 00:23:01,600 people ask me about, oh, you know, what is testing 495 00:23:01,600 --> 00:23:03,320 about? And why do you care? I 496 00:23:03,328 --> 00:23:05,560 always go with that. That's a good question. So 497 00:23:05,568 --> 00:23:09,580 of, you know, what flavor of crisps or chips do you 498 00:23:09,580 --> 00:23:12,160 like? And they go, oh, I like ready 499 00:23:12,160 --> 00:23:14,560 salted or pre -cured or something like that. You're 500 00:23:14,560 --> 00:23:16,320 wrong. It's salt and vinegar because 501 00:23:16,320 --> 00:23:19,480 they're my favorite. And I know quality and you 502 00:23:19,480 --> 00:23:21,920 don't have quality. And they're like, no, no, 503 00:23:21,960 --> 00:23:24,440 you're wrong. Like this flavor is that, you 504 00:23:24,440 --> 00:23:27,620 know, we are all individuals and we're all 505 00:23:27,620 --> 00:23:29,680 contextual in our own rights. And 506 00:23:29,680 --> 00:23:31,720 the same thing can kind of be applied against the 507 00:23:31,720 --> 00:23:33,680 products. So when we think about 508 00:23:33,680 --> 00:23:36,740 quality, we have to think of it not necessarily as a 509 00:23:36,740 --> 00:23:39,140 singular thing, but as like this multi -dimensional 510 00:23:39,140 --> 00:23:41,320 thing. So there are different types 511 00:23:41,320 --> 00:23:43,760 of characteristics. So I mentioned some earlier 512 00:23:43,760 --> 00:23:45,940 when I was talking about accuracy and results and 513 00:23:45,940 --> 00:23:49,460 responsiveness, there is a great list of quality 514 00:23:49,460 --> 00:23:52,680 characteristics from test eye, which you can Google 515 00:23:52,680 --> 00:23:57,880 and it must have something like 50, 60, maybe 70 plus 516 00:23:57,880 --> 00:24:01,340 quality characteristics out there and stuff. And 517 00:24:01,340 --> 00:24:04,080 some of them are, you know, technical based. So 518 00:24:04,080 --> 00:24:07,400 does it, is it operable across different devices? 519 00:24:07,440 --> 00:24:09,060 Does it integrate with 520 00:24:09,060 --> 00:24:11,760 environments and stuff? But then some of them are much 521 00:24:11,760 --> 00:24:13,480 more kind of like a motif. So 522 00:24:13,480 --> 00:24:16,560 does it feel good to use? Does 523 00:24:16,560 --> 00:24:20,380 it look good? Does it make me excited? So 524 00:24:20,380 --> 00:24:22,940 lots of different characteristics. Then 525 00:24:22,940 --> 00:24:26,080 you have kind of the time factor of this is that 526 00:24:26,080 --> 00:24:29,720 different things will matter at different times. Another 527 00:24:29,720 --> 00:24:32,460 factor to take a step back is different quality 528 00:24:32,460 --> 00:24:35,220 characteristics will matter to different people. So 529 00:24:35,220 --> 00:24:39,640 our end users might care that it looks good and is 530 00:24:39,640 --> 00:24:42,440 easy to use. But if we're in a regulated 531 00:24:42,440 --> 00:24:45,180 environment, our auditor wants to make sure that 532 00:24:45,180 --> 00:24:48,160 it's got quality characteristics of we can 533 00:24:48,160 --> 00:24:49,720 understand how it's working. It's 534 00:24:49,720 --> 00:24:53,060 got good auditing processes, that sort of thing. So 535 00:24:53,060 --> 00:24:55,140 different people have different perspectives in 536 00:24:55,140 --> 00:24:57,300 that way as well. And those things change over 537 00:24:57,300 --> 00:24:59,760 time. So if you're a startup, what 538 00:24:59,760 --> 00:25:02,100 quality means to your end users and to your 539 00:25:02,100 --> 00:25:04,100 stakeholders is going to mean something very 540 00:25:04,100 --> 00:25:06,260 different when you're moving into like a small medium 541 00:25:06,260 --> 00:25:11,100 company to enterprise. So we constantly have to keep 542 00:25:11,100 --> 00:25:14,080 asking ourselves regularly, like what does quality mean 543 00:25:14,080 --> 00:25:17,080 to our end users to the people that matter? And 544 00:25:17,080 --> 00:25:19,740 what does it matter like over time? Otherwise, 545 00:25:20,000 --> 00:25:22,600 again, the quality is connected to the risks and 546 00:25:22,600 --> 00:25:24,820 the risks is connected to the testing. If 547 00:25:24,820 --> 00:25:28,060 we don't keep an eye on quality, we again are 548 00:25:28,060 --> 00:25:31,980 testing and our development will drift because we are 549 00:25:31,980 --> 00:25:33,900 no longer building the thing that the people that 550 00:25:33,900 --> 00:25:36,360 matter care about anymore. So 551 00:25:36,360 --> 00:25:39,060 yeah, that's kind of why we want to think about 552 00:25:39,060 --> 00:25:42,200 quality characteristics on a regular basis. Yeah, 553 00:25:42,220 --> 00:25:45,480 some people actually point the terms quality attributes 554 00:25:45,480 --> 00:25:48,180 here like non -functional requirements or performance 555 00:25:48,180 --> 00:25:51,700 attributes or there are multiple terms about it. 556 00:25:51,820 --> 00:25:53,360 But essentially, you're looking 557 00:25:53,360 --> 00:25:55,980 into like the elities. So some people also call it 558 00:25:55,980 --> 00:25:58,780 elities, availability, scalability and things like 559 00:25:58,780 --> 00:26:00,560 that. So I think this quality 560 00:26:00,560 --> 00:26:03,220 attributes sometimes in some teams, they tend to under 561 00:26:03,220 --> 00:26:05,700 prioritize that and only focus on the functional 562 00:26:05,700 --> 00:26:08,120 aspect. Like given certain input, what's 563 00:26:08,120 --> 00:26:10,200 the output that should come out? And 564 00:26:10,200 --> 00:26:13,140 they just create more test cases around that. But 565 00:26:13,140 --> 00:26:15,680 actually, when you have holistic approach, so you 566 00:26:15,680 --> 00:26:18,200 will have aspects of quality attributes that you 567 00:26:18,200 --> 00:26:20,100 want to test as well. And again, depending on the 568 00:26:20,100 --> 00:26:22,840 product, depending on the kind of quality that the 569 00:26:22,840 --> 00:26:25,940 team or the user cares about, you build some kind 570 00:26:25,940 --> 00:26:28,940 of testing strategies around those things as well. So 571 00:26:28,940 --> 00:26:31,140 I think it's really important for those of you 572 00:26:31,140 --> 00:26:33,620 who would love to ascertain your quality of the 573 00:26:33,620 --> 00:26:36,080 product, first of all, know about your quality 574 00:26:36,080 --> 00:26:38,980 attributes, quality characteristics, and then 575 00:26:38,980 --> 00:26:42,100 build some kind of testing strategies around those. So 576 00:26:42,100 --> 00:26:45,660 let's maybe go into several testing strategies that you 577 00:26:45,660 --> 00:26:47,920 cover in the book. Maybe some from the 578 00:26:47,920 --> 00:26:50,860 implementation, some from the imagination. And 579 00:26:50,860 --> 00:26:52,720 later on, we'll also talk about automation testing, 580 00:26:52,720 --> 00:26:55,580 definitely. So the first strategy that I 581 00:26:55,580 --> 00:26:57,920 want to pick is actually testing API design. 582 00:26:58,340 --> 00:27:00,680 Personally, I find this a bit rare 583 00:27:00,680 --> 00:27:03,920 being covered in the development space or at 584 00:27:03,920 --> 00:27:06,200 least in my area. So what do you mean by 585 00:27:06,200 --> 00:27:08,900 testing API design? How do we do that? And 586 00:27:08,900 --> 00:27:11,340 why is it important? So I think it's interesting 587 00:27:11,340 --> 00:27:13,720 that you say that you don't see that very often, 588 00:27:13,820 --> 00:27:17,220 because I think it happens all the time, but it's not 589 00:27:17,220 --> 00:27:21,740 done in a explicit, structured, or clear format. 590 00:27:22,240 --> 00:27:24,300 So I've worked with developers 591 00:27:24,300 --> 00:27:27,660 and they're doing this stuff implicitly. They're 592 00:27:27,660 --> 00:27:29,220 asking those questions as well. And 593 00:27:29,220 --> 00:27:31,100 that is a form of testing. So 594 00:27:31,100 --> 00:27:33,500 when we're talking about like testing API designs, 595 00:27:33,500 --> 00:27:36,080 this is where this whole sort of shift left mindset 596 00:27:36,080 --> 00:27:38,700 comes in. Again, it's that testing the 597 00:27:38,700 --> 00:27:41,580 ideas, testing the assumptions. So 598 00:27:41,580 --> 00:27:45,320 we're in a situation, perhaps, or doing something 599 00:27:45,320 --> 00:27:48,320 like a collaborative session with different people. So 600 00:27:48,320 --> 00:27:51,280 that's all kind of three Amigo style things or just 601 00:27:51,280 --> 00:27:53,760 again, something informal, but we're together as a 602 00:27:53,760 --> 00:27:57,060 team and we're looking to implement something. So 603 00:27:57,060 --> 00:27:59,660 what we're thinking here is what is it that we want 604 00:27:59,660 --> 00:28:02,060 to build? What is the solution that we're 605 00:28:02,060 --> 00:28:04,660 proposing? And then it's asking questions 606 00:28:04,660 --> 00:28:08,320 about aspects of it. So you're asking questions 607 00:28:08,320 --> 00:28:10,860 around maybe the actual sort of technical 608 00:28:10,860 --> 00:28:13,400 implementation. Maybe you're asking questions 609 00:28:13,400 --> 00:28:16,000 around, oh, you've set these business rules. Do 610 00:28:16,000 --> 00:28:19,040 you mean like, is the boundary here or is it 611 00:28:19,040 --> 00:28:21,380 there? What do you mean by this sort 612 00:28:21,380 --> 00:28:23,920 of domain phrase? I love it when someone says, 613 00:28:23,940 --> 00:28:25,720 I just want to ask a stupid question because 614 00:28:25,720 --> 00:28:28,160 they're the best questions because they're usually 615 00:28:28,160 --> 00:28:30,420 because someone's like, I don't understand an aspect 616 00:28:30,420 --> 00:28:32,040 of this. So I need clarification. 617 00:28:32,500 --> 00:28:34,320 There's nothing more satisfying when 618 00:28:34,320 --> 00:28:37,060 you ask that three people answer the different answers 619 00:28:37,060 --> 00:28:41,660 because everyone's got that sort of misunderstanding. So 620 00:28:41,660 --> 00:28:44,700 it's about understanding the implementation, raising 621 00:28:44,700 --> 00:28:47,700 risks, raising test ideas and stuff. So, 622 00:28:47,860 --> 00:28:50,220 oh, I see that you want to put a limit on this 623 00:28:50,220 --> 00:28:52,300 field. What happens if I go over that 624 00:28:52,300 --> 00:28:55,380 limit and they go, well, you know, you should return 625 00:28:55,380 --> 00:28:58,120 a 500 status code. Shouldn't 626 00:28:58,120 --> 00:29:00,840 it be like a 400 because it's a user input error 627 00:29:00,840 --> 00:29:02,880 and stuff and it's conversations leads to that. 628 00:29:03,540 --> 00:29:05,080 So, yeah, like testing API 629 00:29:05,080 --> 00:29:08,200 designs is very much about just making that process 630 00:29:08,200 --> 00:29:12,000 that everybody does, making it more explicit, making it 631 00:29:12,000 --> 00:29:14,500 more collaborative. So getting people to share 632 00:29:14,500 --> 00:29:17,460 ideas so that we come out, we have a clear idea of 633 00:29:17,460 --> 00:29:20,780 what it is that we want to build and also that we 634 00:29:20,780 --> 00:29:22,500 are all on the same page. Like 635 00:29:22,500 --> 00:29:24,600 we all agree. I remember seeing some 636 00:29:24,600 --> 00:29:26,680 research once. I wish I could find it, but 637 00:29:26,680 --> 00:29:30,040 there was research that showed that most bugs come 638 00:29:30,040 --> 00:29:33,520 not from errors in code, but from misunderstandings 639 00:29:33,520 --> 00:29:36,540 of requirements or of user's needs. So 640 00:29:36,540 --> 00:29:38,480 I think that's why this sort of aspect is 641 00:29:38,480 --> 00:29:40,740 essential. And it's useful for a testing 642 00:29:40,740 --> 00:29:43,060 perspective as well because you can ask the questions 643 00:29:43,060 --> 00:29:46,540 then of, how are we going to test this and what 644 00:29:46,540 --> 00:29:48,580 things do we need to do to make this thing 645 00:29:48,580 --> 00:29:50,960 testable. And then, yeah, you can factor 646 00:29:50,960 --> 00:29:52,900 in different types of tools. So 647 00:29:52,900 --> 00:29:54,940 like things like swagger documentation is really 648 00:29:54,940 --> 00:29:58,980 useful because it's a way of documenting the API. You 649 00:29:58,980 --> 00:30:01,560 know, when it makes it readable, you can trigger 650 00:30:01,560 --> 00:30:04,720 off ideas and questions from and conversations from, 651 00:30:05,060 --> 00:30:08,340 but it's not implementing the whole code base. And 652 00:30:08,340 --> 00:30:11,620 then it's got its own sort of ecosystem of tools to 653 00:30:11,620 --> 00:30:15,180 make it easier for people to translate that into code 654 00:30:15,180 --> 00:30:17,320 at a later date. So you get a kind of double 655 00:30:17,320 --> 00:30:20,800 win there of everybody's on the same page. We've 656 00:30:20,800 --> 00:30:23,140 got this explicit documentation, press of a 657 00:30:23,140 --> 00:30:26,440 button, attention to turn this, get us halfway down 658 00:30:26,440 --> 00:30:29,200 the line in terms of implementing a new endpoint 659 00:30:29,200 --> 00:30:32,420 or a new API. Right. So I think this approach, 660 00:30:32,500 --> 00:30:34,820 when you mentioned about collaborative activities, 661 00:30:35,180 --> 00:30:38,120 right, I think again, it's rare to me always, you 662 00:30:38,120 --> 00:30:40,540 know, seeing the three Amigos in practice, right, 663 00:30:40,620 --> 00:30:42,460 like maybe the business analyst or the product 664 00:30:42,460 --> 00:30:45,260 owner, right, with the tester and developer, always 665 00:30:45,260 --> 00:30:49,160 collaborating actively for almost all of the stories 666 00:30:49,160 --> 00:30:50,820 or, you know, the requirements that they're 667 00:30:50,820 --> 00:30:52,960 implementing. So I think the key again here 668 00:30:52,960 --> 00:30:55,780 just to explicitly mention it, collaborative software 669 00:30:55,780 --> 00:30:57,840 development, I think is really, really crucial, 670 00:30:58,000 --> 00:31:00,260 right, because a shared understanding, you don't 671 00:31:00,260 --> 00:31:02,560 want to misunderstand the software requirements or 672 00:31:02,560 --> 00:31:05,140 expected behavior of the product that you're 673 00:31:05,140 --> 00:31:07,480 building, right. So do a lot more of these 674 00:31:07,480 --> 00:31:09,740 testing API design, right. Maybe 675 00:31:09,740 --> 00:31:12,000 it's not just API design, sometimes it's driven by 676 00:31:12,000 --> 00:31:14,440 the UI, right, for people who are building UI -centric 677 00:31:14,440 --> 00:31:16,780 journey. And I saw talk about it from 678 00:31:16,780 --> 00:31:20,360 the API design perspective, but this is a mindset and 679 00:31:20,360 --> 00:31:22,300 approach that we've applied to our team in general. 680 00:31:22,740 --> 00:31:24,080 And like, it's something that's 681 00:31:24,080 --> 00:31:26,920 been championed by Lisa Crispin and Janet Gregory, 682 00:31:27,340 --> 00:31:29,860 their agile testing books, all of their work on 683 00:31:29,860 --> 00:31:31,700 continuous testing as well. And 684 00:31:31,700 --> 00:31:33,940 they're big advocates for holistic testing as well. 685 00:31:34,400 --> 00:31:35,640 But they've really sort of kind 686 00:31:35,640 --> 00:31:38,860 of opened up that mindset of getting teams to talk 687 00:31:38,860 --> 00:31:40,880 to each other about the things that they're 688 00:31:40,880 --> 00:31:44,460 building, the quality that they want to imbue into 689 00:31:44,460 --> 00:31:46,200 the things that they build. So 690 00:31:46,200 --> 00:31:48,300 yeah, you can apply it to any sort of kind of 691 00:31:48,300 --> 00:31:50,300 context that you're working in. Yeah. 692 00:31:50,380 --> 00:31:52,640 And not to mention also, after you get that shared 693 00:31:52,640 --> 00:31:55,200 understanding, please write it down somewhere, maybe as 694 00:31:55,200 --> 00:31:57,340 a documentation or maybe some kind of a test 695 00:31:57,340 --> 00:31:59,520 specifications, whatever that is, right, so that the 696 00:31:59,520 --> 00:32:02,400 shared understanding is not lost when people change or 697 00:32:02,400 --> 00:32:04,860 maybe business strategy change and things like 698 00:32:04,860 --> 00:32:06,760 that. So thanks for sharing about 699 00:32:06,760 --> 00:32:09,660 this testing API design. So the second testing strategy 700 00:32:09,660 --> 00:32:11,860 that I'd like you to maybe talk a little bit more, 701 00:32:12,000 --> 00:32:14,080 which is what you mentioned in the beginning, something 702 00:32:14,080 --> 00:32:17,460 that you have passion about exploratory testing. So 703 00:32:17,460 --> 00:32:20,140 what is exploratory testing and how can we approach 704 00:32:20,140 --> 00:32:24,480 this kind of testing? So exploratory testing for me 705 00:32:24,480 --> 00:32:28,700 is a semi -structured approach to testing. It 706 00:32:28,700 --> 00:32:32,360 basically relies on my creativity as an individual, 707 00:32:32,360 --> 00:32:34,720 but still has enough structure and enough 708 00:32:34,720 --> 00:32:38,460 boundary and place that we are focused on what we 709 00:32:38,460 --> 00:32:42,000 want to do. So I very much enjoy sort of 710 00:32:42,000 --> 00:32:45,980 charter -based approach to exploratory testing, which 711 00:32:45,980 --> 00:32:48,760 I've mentioned a lot in Exploratory by Elizabeth 712 00:32:48,760 --> 00:32:51,260 Hendrickson. And the idea is, again, so 713 00:32:51,260 --> 00:32:54,620 it's connected to risk. If I identify the risk, I set 714 00:32:54,620 --> 00:32:57,720 out an exploratory testing charter to learn about that 715 00:32:57,720 --> 00:33:00,920 risk, and then I will do testing. And 716 00:33:00,920 --> 00:33:04,060 it's not scripted. It is very much me following 717 00:33:04,060 --> 00:33:08,560 my instincts, using my own internal heuristics, maybe 718 00:33:08,560 --> 00:33:11,200 using some external heuristics as well, taking 719 00:33:11,200 --> 00:33:14,540 notes, reviewing those and looking at sort of paths 720 00:33:14,540 --> 00:33:18,120 and avenues towards how I will do my testing. So 721 00:33:18,120 --> 00:33:21,220 it's not great for repeatability, but it's 722 00:33:21,220 --> 00:33:24,480 really good for sort of expansive broad testing 723 00:33:24,480 --> 00:33:27,720 that's going to reach into areas that scripted testing 724 00:33:27,720 --> 00:33:30,000 is not really going to work on. One 725 00:33:30,000 --> 00:33:32,600 of the main reasons why I love exploratory testing as 726 00:33:32,600 --> 00:33:35,260 well is if you're a good exploratory tester, it 727 00:33:35,260 --> 00:33:38,140 starts to blur the edges of what automation means. 728 00:33:38,520 --> 00:33:39,640 Some of the most successful 729 00:33:39,640 --> 00:33:42,500 things I've had in the automation space have been 730 00:33:42,500 --> 00:33:44,700 within the context of exploratory testing. So 731 00:33:44,708 --> 00:33:47,600 yeah, it's great building automated test cases and 732 00:33:47,600 --> 00:33:50,420 test scripts and stuff, but if you can build a tool 733 00:33:50,420 --> 00:33:53,900 that can scrape your monitoring system or your 734 00:33:53,900 --> 00:33:56,640 log files to let you know when errors occur, if you 735 00:33:56,640 --> 00:33:59,140 can build a little script that injects all of the 736 00:33:59,140 --> 00:34:02,120 fields, like if you've got a massive like 20 input 737 00:34:02,120 --> 00:34:05,100 form field to fill in, but you only need to test one 738 00:34:05,100 --> 00:34:07,900 field or something like that, building a tool that 739 00:34:07,900 --> 00:34:10,679 can inject data into that, tools that can set up data 740 00:34:10,679 --> 00:34:13,960 for you really quickly. I find that quite satisfying 741 00:34:13,960 --> 00:34:17,460 as an activity, and it allows me to accelerate my 742 00:34:17,460 --> 00:34:20,380 exploratory testing because I'm less focused on the 743 00:34:20,380 --> 00:34:24,080 setup, than more into the sort of observation and 744 00:34:24,080 --> 00:34:27,300 analysis and reaction to those sort of things as 745 00:34:27,300 --> 00:34:29,360 well. I think the charter is very 746 00:34:29,360 --> 00:34:31,679 important when you do this exploratory testing. 747 00:34:32,699 --> 00:34:34,540 Exploratory doesn't mean that it's 748 00:34:34,540 --> 00:34:36,659 super, you know, like freedom, right? You 749 00:34:36,659 --> 00:34:39,199 can just do whatever you like and figure it out if 750 00:34:39,199 --> 00:34:40,920 you find any bugs or not. But 751 00:34:40,920 --> 00:34:43,360 I think the most probably more effective way is 752 00:34:43,360 --> 00:34:45,219 actually to set a charter, right? Again, 753 00:34:45,300 --> 00:34:48,120 it's driven by the risk that you identify earlier 754 00:34:48,120 --> 00:34:50,060 regarding your quality attributes and things like 755 00:34:50,060 --> 00:34:51,800 that. And then you kind of like set 756 00:34:51,800 --> 00:34:55,139 the charter, you explore based on that theme and 757 00:34:55,139 --> 00:34:56,739 see how it goes, right? And 758 00:34:56,739 --> 00:34:59,500 some people and some teams actually do like bug 759 00:34:59,500 --> 00:35:02,740 bashing, you know, like group activities where, you 760 00:35:02,740 --> 00:35:04,980 know, you set maybe a theme, then they will just 761 00:35:04,980 --> 00:35:08,420 explore the application, try to break it and report 762 00:35:08,420 --> 00:35:11,020 issues that they find along the way. So 763 00:35:11,020 --> 00:35:12,920 I think for people who would love to do 764 00:35:12,920 --> 00:35:15,360 exploratory testing, so one tip here, very, very 765 00:35:15,360 --> 00:35:18,000 important is to set it based on the quality 766 00:35:18,000 --> 00:35:21,180 characteristics and also set a charter before you start 767 00:35:21,180 --> 00:35:24,260 it. Yeah. And so I wondered, I had 768 00:35:24,260 --> 00:35:27,180 one last thing to that as well is that you can go 769 00:35:27,180 --> 00:35:30,740 off charter as well. So if you find something like 770 00:35:30,740 --> 00:35:33,620 some astonishingly bad part of the system elsewhere, 771 00:35:33,620 --> 00:35:37,440 there's nothing wrong with going off charter as well. 772 00:35:37,860 --> 00:35:39,260 The important fact is that you 773 00:35:39,260 --> 00:35:41,180 know that you've done that. And 774 00:35:41,180 --> 00:35:43,100 then you can be like, right, well, I've discovered 775 00:35:43,100 --> 00:35:45,160 all this other interesting stuff. Now 776 00:35:45,160 --> 00:35:47,420 I'm going to go back to the thing that I wanted to 777 00:35:47,420 --> 00:35:51,100 focus on. Whereas I will go look at the 778 00:35:51,100 --> 00:35:53,220 other thing and then go, my work here is done and 779 00:35:53,220 --> 00:35:55,880 forget about the initial thing that we were working. 780 00:35:56,460 --> 00:35:58,560 So again, like risk is your 781 00:35:58,560 --> 00:36:03,880 guide, but it doesn't set strict boundaries on what 782 00:36:03,880 --> 00:36:06,620 you can and can't do. You do have flexibility and 783 00:36:06,620 --> 00:36:08,760 fluency in it as well. Right. 784 00:36:09,020 --> 00:36:11,240 Thanks for adding that. So I think you can go off 785 00:36:11,240 --> 00:36:12,600 chart. But yeah, don't forget to, 786 00:36:12,600 --> 00:36:14,540 again, document your findings, right? Don't 787 00:36:14,540 --> 00:36:16,980 just take it like ad hoc activity and that's it, 788 00:36:17,040 --> 00:36:20,060 right? So speaking about testing, you 789 00:36:20,067 --> 00:36:22,900 know, obviously people associate that a lot with 790 00:36:22,900 --> 00:36:25,260 the automated API testing, right? Sometimes 791 00:36:25,260 --> 00:36:27,580 it's called end to end testing or acceptance 792 00:36:27,580 --> 00:36:30,400 testing. Maybe tell us more how can we 793 00:36:30,400 --> 00:36:32,840 do better automated testing because I'm sure many 794 00:36:32,840 --> 00:36:34,840 people understand about automated testing, but how 795 00:36:34,840 --> 00:36:37,900 to do it more effectively or better is something that 796 00:36:37,900 --> 00:36:40,680 maybe you can give some advice on. Yeah. 797 00:36:40,880 --> 00:36:42,980 So it's interesting, like sort of drawing the 798 00:36:42,980 --> 00:36:44,860 parallels between end to end testing and automation 799 00:36:44,860 --> 00:36:47,320 testing because I think that automation is literally 800 00:36:47,320 --> 00:36:50,160 just using tools to do something that we were 801 00:36:50,160 --> 00:36:52,860 doing. So one of the big things, 802 00:36:52,920 --> 00:36:57,500 again, with automation is focusing on risks. So 803 00:36:57,500 --> 00:37:00,640 I have these acronyms or these sort of phrases that 804 00:37:00,640 --> 00:37:04,360 I use regularly to sort of help me understand what I'm 805 00:37:04,360 --> 00:37:07,120 automating. I have this tutu and Tata. 806 00:37:07,480 --> 00:37:08,760 So from the user interface, am 807 00:37:08,760 --> 00:37:12,700 I testing the UI or am I testing through the UI? A 808 00:37:12,700 --> 00:37:14,840 lot of people that work in the automation space, 809 00:37:14,940 --> 00:37:18,840 especially testers, will do things full stack and 810 00:37:18,840 --> 00:37:21,140 they've got some automated tests running, but the 811 00:37:21,140 --> 00:37:23,300 automated test is actually focused on a risk in the 812 00:37:23,300 --> 00:37:25,920 back end, like some business calculation. You 813 00:37:25,920 --> 00:37:28,060 don't need the user interface for that. You 814 00:37:28,060 --> 00:37:30,860 could do on the API layer, which is why we have Tata. 815 00:37:31,140 --> 00:37:32,760 Am I testing the API or am I 816 00:37:32,760 --> 00:37:34,700 testing through the API? Again, 817 00:37:34,700 --> 00:37:37,560 if this is some sort of business calculation, some 818 00:37:37,560 --> 00:37:39,980 sort of rule, then couldn't I build a unit test for 819 00:37:39,980 --> 00:37:42,280 this? So I think sometimes like the 820 00:37:42,280 --> 00:37:45,020 success with API automation or success with any 821 00:37:45,020 --> 00:37:48,380 automation is almost the inverse of saying, I'm not 822 00:37:48,380 --> 00:37:50,800 going to automate it on this layer. I'm 823 00:37:50,800 --> 00:37:53,100 not going to automate that thing. I'm 824 00:37:53,100 --> 00:37:55,100 going to look at talking about end to end tests. 825 00:37:55,220 --> 00:37:56,760 I'll look at all the different 826 00:37:56,760 --> 00:37:59,840 parts of the system that I'm touching with my end 827 00:37:59,840 --> 00:38:02,620 to end test and I'll try and be more targeted. 828 00:38:02,620 --> 00:38:04,980 However, then what I mean is the 829 00:38:04,980 --> 00:38:07,540 risks that we do care about on the API are much 830 00:38:07,540 --> 00:38:10,800 more focused around like how it's presented, how 831 00:38:10,800 --> 00:38:14,020 it's structured. Does it respond in the correct 832 00:38:14,020 --> 00:38:17,020 way that we want it to respond? Contract 833 00:38:17,020 --> 00:38:19,700 testing sits under the banner of automated testing 834 00:38:19,700 --> 00:38:22,220 because it's using tools again. That's 835 00:38:22,220 --> 00:38:24,460 very much focused on contract drift between 836 00:38:24,460 --> 00:38:26,900 integrations as well. Performance 837 00:38:26,900 --> 00:38:30,240 testing is technically automated testing as well. 838 00:38:30,800 --> 00:38:32,180 The oracle is slightly different 839 00:38:32,180 --> 00:38:34,100 in terms of how we determine what's good or 840 00:38:34,100 --> 00:38:37,900 bad, but we still need to use tools to put an API 841 00:38:37,900 --> 00:38:40,900 or APIs under load as well. So, 842 00:38:41,140 --> 00:38:43,780 you know, Stein -Salica stuck record, but again, 843 00:38:43,840 --> 00:38:46,080 it's all about risk. Like going back to what I was 844 00:38:46,087 --> 00:38:48,700 saying earlier with being more specific and targeted, 845 00:38:49,280 --> 00:38:53,320 if I have tested every individual component within 846 00:38:53,320 --> 00:38:57,860 my API with unit tests, then my API tests are much 847 00:38:57,860 --> 00:39:00,980 more about do these things integrate, might receiving 848 00:39:00,980 --> 00:39:03,140 the right requests and sending the right responses. 849 00:39:03,840 --> 00:39:05,500 So, what you end up with is 850 00:39:05,500 --> 00:39:08,480 probably less API tests. Again, 851 00:39:08,560 --> 00:39:10,960 context matters here. I've worked on projects where I 852 00:39:10,960 --> 00:39:14,240 can't go to unit tests because everything already 853 00:39:14,240 --> 00:39:16,880 comes to us compiled. And I don't just mean me as a 854 00:39:16,880 --> 00:39:18,920 tester. I mean, literally, the whole 855 00:39:18,920 --> 00:39:21,740 company gets given an application that's already 856 00:39:21,740 --> 00:39:24,300 built and then we're adding onto it. So, 857 00:39:24,360 --> 00:39:26,880 we have to do everything on the API layer, but 858 00:39:26,880 --> 00:39:30,160 we're making that informed decision based on what are 859 00:39:30,160 --> 00:39:32,820 the risks, the context as well. So, 860 00:39:33,000 --> 00:39:35,160 I think, yeah, a big thing with automation is that 861 00:39:35,160 --> 00:39:37,680 you're not trying to be exhaustive on the API 862 00:39:37,680 --> 00:39:40,060 layer. You're trying to be selective in 863 00:39:40,060 --> 00:39:42,120 terms of, again, the types of risks that we care 864 00:39:42,120 --> 00:39:46,260 about and assume that other testing activities are going 865 00:39:46,260 --> 00:39:49,520 on, maybe on the more atomic level. So, 866 00:39:49,580 --> 00:39:52,880 bringing it back to end to end testing, what my goal 867 00:39:52,880 --> 00:39:55,320 for an end to end test is, does everything work 868 00:39:55,320 --> 00:39:58,200 end to end? That's literally all my test does. 869 00:39:58,380 --> 00:39:59,920 Does this bit, the front end, 870 00:40:00,080 --> 00:40:03,840 talk to the back end? Does API talk to API B, C and 871 00:40:03,840 --> 00:40:05,800 D? But I don't really care about 872 00:40:05,800 --> 00:40:07,900 the business logic in them because I know that's been 873 00:40:07,900 --> 00:40:11,320 checked or tested in a previous way. I 874 00:40:11,320 --> 00:40:13,200 like one heuristic that you mentioned in the book, 875 00:40:13,300 --> 00:40:14,860 right? So, if you have the capability 876 00:40:14,860 --> 00:40:18,120 to push your checks or your testing lower, you 877 00:40:18,120 --> 00:40:20,320 should actually try to do that more often, right? 878 00:40:20,440 --> 00:40:22,180 Because if you're implemented in 879 00:40:22,180 --> 00:40:23,940 the upper layer, most likely, it's going to be 880 00:40:23,940 --> 00:40:26,940 slower, less maintainable, and also more complicated 881 00:40:26,940 --> 00:40:29,480 to set up, right? So, I think this is very 882 00:40:29,480 --> 00:40:32,620 important for people not just focus a lot on the 883 00:40:32,620 --> 00:40:35,720 end to end testing or API testing, the 2 -2 -tata 884 00:40:35,720 --> 00:40:37,800 thing, I think that's also very important. Are 885 00:40:37,800 --> 00:40:40,100 you testing the API itself? Or 886 00:40:40,100 --> 00:40:42,700 are you testing through the API, which is kind of like 887 00:40:42,700 --> 00:40:45,980 the business logic or the validation and things like 888 00:40:45,980 --> 00:40:47,640 that, right? So, I think that's kind of 889 00:40:47,640 --> 00:40:49,640 like a good heuristic as well that people can use 890 00:40:49,640 --> 00:40:52,020 so that we can have a more holistic approach to 891 00:40:52,020 --> 00:40:54,200 testing. And I think I also like 892 00:40:54,200 --> 00:40:56,360 another thing that you mentioned in the book about 893 00:40:56,360 --> 00:40:59,560 automation testing, because many teams actually drive 894 00:40:59,560 --> 00:41:03,580 the API testing by coverage or just user requirements. 895 00:41:04,020 --> 00:41:05,060 So, again, I think you 896 00:41:05,060 --> 00:41:07,660 mentioned it several times already, use risk -based 897 00:41:07,660 --> 00:41:10,620 approach, don't just focus on the coverage, right? You 898 00:41:10,627 --> 00:41:12,900 know, like aiming for higher code coverage or 899 00:41:12,900 --> 00:41:15,560 test coverage, right? But also look at the different 900 00:41:15,560 --> 00:41:17,780 type of risks that you cover. I 901 00:41:17,780 --> 00:41:19,520 think that's very important. I 902 00:41:19,520 --> 00:41:21,760 was going to say, like coverage of risk is a type 903 00:41:21,760 --> 00:41:24,400 of coverage. What we're trying to do here is 904 00:41:24,400 --> 00:41:28,960 not necessarily cover every instance of data, every 905 00:41:28,960 --> 00:41:32,400 code path, or, you know, different types of coverage, 906 00:41:32,580 --> 00:41:34,520 like, you know, is some tool reporting factor. 907 00:41:35,760 --> 00:41:37,080 You've mentioned it, but how has 908 00:41:37,080 --> 00:41:38,880 it decided that number? There's 909 00:41:38,880 --> 00:41:41,520 nothing wrong with the word coverage. It's 910 00:41:41,520 --> 00:41:44,000 more about what are you covering? That's 911 00:41:44,000 --> 00:41:46,380 why, yeah, I think what we should be covering is the 912 00:41:46,380 --> 00:41:49,880 risks that matter to us, not necessarily the amount 913 00:41:49,880 --> 00:41:52,780 of paths we have for what some sort of tool is 914 00:41:52,780 --> 00:41:55,140 telling us is acceptable. Right. 915 00:41:55,540 --> 00:41:58,160 So, maybe speaking a little bit on the requirements 916 00:41:58,160 --> 00:42:00,400 aspect, right? Because you mentioned maybe a lot 917 00:42:00,400 --> 00:42:03,380 of bugs are introduced by a lot of misunderstanding 918 00:42:03,380 --> 00:42:06,880 about the requirements. So, how can we get less 919 00:42:06,880 --> 00:42:09,620 misunderstanding? What about this acceptance driven 920 00:42:09,620 --> 00:42:12,240 testing that some people advocate about? Do 921 00:42:12,240 --> 00:42:15,200 you have some advice on this area as well? Yeah. 922 00:42:15,320 --> 00:42:17,640 So, it's an interesting space, like acceptance test 923 00:42:17,640 --> 00:42:20,500 driven design and like behavior driven design as 924 00:42:20,500 --> 00:42:22,680 well. It took me a long time to 925 00:42:22,680 --> 00:42:25,860 sort of get my head around it, really. You 926 00:42:25,860 --> 00:42:28,160 have to kind of break it down into smaller chunks. 927 00:42:28,360 --> 00:42:29,060 So, we talked about like 928 00:42:29,060 --> 00:42:31,640 behavior driven design or behavior driven development. 929 00:42:32,160 --> 00:42:33,100 We've already kind of talked 930 00:42:33,100 --> 00:42:35,620 about one of the core tenants of BDD already, 931 00:42:36,020 --> 00:42:40,560 which is shift left testing ideas, questioning designs, 932 00:42:41,080 --> 00:42:43,540 doing that work together collaboratively. So, 933 00:42:43,580 --> 00:42:47,560 we all have that understanding from there, as 934 00:42:47,560 --> 00:42:49,840 you were mentioning earlier as well, then we want to 935 00:42:49,840 --> 00:42:53,160 capture that into some sort of documentation, some sort 936 00:42:53,160 --> 00:42:56,680 of kind of concrete examples and scenarios that 937 00:42:56,680 --> 00:42:59,640 describe how we expect things to work. And 938 00:42:59,640 --> 00:43:01,540 then it's from there that we can use that for 939 00:43:01,540 --> 00:43:03,480 acceptance test driven design. I 940 00:43:03,480 --> 00:43:05,680 always think that it's really interesting with ATB. 941 00:43:06,140 --> 00:43:08,360 It's because the way that the 942 00:43:08,360 --> 00:43:12,100 risks that I say that that's mitigating aren't 943 00:43:12,100 --> 00:43:16,960 functional business risks, it's more team risks. So, 944 00:43:17,020 --> 00:43:19,640 yeah, I was running a workshop on automation and 945 00:43:19,640 --> 00:43:23,220 this tester who was there turned around and sort of 946 00:43:23,220 --> 00:43:25,220 thought they'd ask a cheeky question. They 947 00:43:25,220 --> 00:43:26,460 said, oh, you know, it's all well and good me 948 00:43:26,460 --> 00:43:28,620 automating all this testing, but can I automate my 949 00:43:28,620 --> 00:43:31,440 developers so they actually build the right thing? And 950 00:43:31,440 --> 00:43:32,760 I was like, well, it's fine. You 951 00:43:32,760 --> 00:43:34,780 should say that because that is the risk that I 952 00:43:34,780 --> 00:43:37,600 think like ATB is mitigating. If 953 00:43:37,600 --> 00:43:41,380 you follow that sort of red green refactor approach 954 00:43:41,380 --> 00:43:44,820 with acceptance test driven design, what it's doing is 955 00:43:44,820 --> 00:43:47,440 it's putting the boundaries in the place to deliver 956 00:43:47,440 --> 00:43:49,580 the right thing. So, it's giving some sort of 957 00:43:49,580 --> 00:43:53,140 cue to the developer of you haven't built what has 958 00:43:53,140 --> 00:43:56,860 been asked of you until you make it pass. But 959 00:43:56,860 --> 00:43:59,680 it's done in a way that because it's described from 960 00:43:59,680 --> 00:44:02,760 a business level or from how a user will interact 961 00:44:02,760 --> 00:44:05,320 with the system, because it's done from that sort 962 00:44:05,320 --> 00:44:08,940 of that level, it still gives the developer scope 963 00:44:08,940 --> 00:44:11,740 to implement it in the way that they want to implement 964 00:44:11,740 --> 00:44:14,700 it. So, it's not so prescribed that 965 00:44:14,700 --> 00:44:18,060 all kind of creativity in the development space is 966 00:44:18,060 --> 00:44:20,040 gone. I like to think of it as 967 00:44:20,040 --> 00:44:22,960 like putting the barriers on when you're bowling, you 968 00:44:22,960 --> 00:44:25,060 know, it stops you from bowling a gutter ball, but 969 00:44:25,060 --> 00:44:28,760 you can still hit one pit or ten as a developer. So, 970 00:44:29,100 --> 00:44:31,600 again, it's this holistic thing going on here is 971 00:44:31,600 --> 00:44:34,880 that ATDD is really useful for those risks of making 972 00:44:34,880 --> 00:44:37,300 sure that we deliver the right things. That 973 00:44:37,300 --> 00:44:40,000 doesn't mean that it's going to cover all of the 974 00:44:40,000 --> 00:44:43,560 other risks by data risks and structural risks. And 975 00:44:43,560 --> 00:44:46,240 that's why we might have other automation or other 976 00:44:46,240 --> 00:44:48,440 testing activities in place. So, 977 00:44:48,560 --> 00:44:51,520 yeah, I'm a big fan of ATDD, but sometimes it gets 978 00:44:51,520 --> 00:44:56,360 kind of a bit lost in the testing soup as a whole. 979 00:44:56,800 --> 00:44:58,000 And, you know, you start seeing 980 00:44:58,000 --> 00:45:00,920 people start using these frameworks to try and 981 00:45:00,920 --> 00:45:04,400 exhaustively test everything or exhaustively set all 982 00:45:04,400 --> 00:45:06,880 these tests out there, developers to follow. And 983 00:45:06,880 --> 00:45:09,320 I think that's where you get fatigue and people stop 984 00:45:09,320 --> 00:45:11,300 using these things because you just end up with too 985 00:45:11,300 --> 00:45:13,940 many tests. You're not focused on the risks 986 00:45:13,940 --> 00:45:17,920 of delivery and just end up sort of overwhelming the 987 00:45:17,920 --> 00:45:20,540 team and they lose interest in stop doing that 988 00:45:20,540 --> 00:45:23,840 approach. Yeah, ATDD definitely has a lot 989 00:45:23,840 --> 00:45:26,300 of things to cover. So, you mentioned some important 990 00:45:26,300 --> 00:45:28,500 aspects like it has to be collaborative. That's 991 00:45:28,500 --> 00:45:31,560 the first thing, right? Use examples to specify the 992 00:45:31,560 --> 00:45:34,140 behaviors, right? Also, like the living document 993 00:45:34,140 --> 00:45:36,400 aspect, right? So, I think it's also powerful 994 00:45:36,400 --> 00:45:38,320 in that aspect. You kind of like write down 995 00:45:38,320 --> 00:45:40,800 your requirements, but also evolve it along the way, 996 00:45:40,840 --> 00:45:42,660 right? It's not just a test script, 997 00:45:42,860 --> 00:45:45,060 but it's much more beyond that. So, 998 00:45:45,080 --> 00:45:47,660 I think for people who love ATDD, do check it 999 00:45:47,660 --> 00:45:48,980 out. I also covered previous 1000 00:45:48,980 --> 00:45:52,000 episodes with John Ferguson and Jan Molak. So, 1001 00:45:52,220 --> 00:45:54,620 speaking about the thing that one of your audience 1002 00:45:54,620 --> 00:45:56,360 asked you just now, right, Chikili, right? How 1003 00:45:56,360 --> 00:45:58,840 to automate developers. I know that you are writing 1004 00:45:58,840 --> 00:46:02,780 a new book in the process, AI Assisted Testing. Maybe 1005 00:46:02,780 --> 00:46:04,660 we can use AI a little bit. So, 1006 00:46:04,680 --> 00:46:06,820 tell us a little bit more snippet, like why are you 1007 00:46:06,820 --> 00:46:09,620 writing this book and what should we expect from AI 1008 00:46:09,620 --> 00:46:12,260 to help us in testing aspect? Yeah, 1009 00:46:12,340 --> 00:46:16,400 so AI Assisted Testing, it's almost complete. It's 1010 00:46:16,400 --> 00:46:19,520 kind of a bit of a logical next step from some 1011 00:46:19,520 --> 00:46:21,240 of the things that I've been talking about. So, 1012 00:46:21,480 --> 00:46:23,280 you know, I was talking about, like, exploratory 1013 00:46:23,280 --> 00:46:26,240 testing and how I use tools within my laboratory 1014 00:46:26,240 --> 00:46:28,260 testing to aid my testing. The 1015 00:46:28,260 --> 00:46:33,500 general crux of the new book is around how we can 1016 00:46:33,500 --> 00:46:37,200 use AI ultimately to assist us to support our testing. 1017 00:46:37,940 --> 00:46:39,000 These things like, you know, 1018 00:46:39,000 --> 00:46:42,480 large language models, Gen AI, they're fantastic. They 1019 00:46:42,480 --> 00:46:45,680 do these amazing things and they've sort of kicked off 1020 00:46:45,680 --> 00:46:47,740 this whole conversation not just for testers, but for 1021 00:46:47,740 --> 00:46:50,400 developers as well of is AI going to replace us. 1022 00:46:51,000 --> 00:46:52,160 But I don't think that's the 1023 00:46:52,160 --> 00:46:53,960 effective way of using them. I 1024 00:46:53,960 --> 00:46:57,180 think the effective way is to reflect on what you do 1025 00:46:57,180 --> 00:47:01,020 in your role and look at the aspects of it that can 1026 00:47:01,020 --> 00:47:04,600 benefit from the use of these types of tools. So, 1027 00:47:04,640 --> 00:47:07,740 for example, things like generating large sets of 1028 00:47:07,740 --> 00:47:10,620 data using large language models for those. In 1029 00:47:10,620 --> 00:47:13,560 the automation space, can I do things like page objects 1030 00:47:13,560 --> 00:47:17,100 and boilerplate code? Can I feed it some HTML and 1031 00:47:17,100 --> 00:47:18,900 then just get my page objects out of it straight 1032 00:47:18,900 --> 00:47:21,560 away? It's exploring the ideas of, 1033 00:47:21,600 --> 00:47:23,900 like, you know, how could these tools potentially be 1034 00:47:23,900 --> 00:47:27,360 trained on our contexts or tuned towards our context 1035 00:47:27,360 --> 00:47:30,040 as well, so that not only when we ask them a 1036 00:47:30,040 --> 00:47:31,900 question, do we get a response, but we get a 1037 00:47:31,900 --> 00:47:34,660 response that is kind of informed by what's going on 1038 00:47:34,660 --> 00:47:36,620 around it? Because a lot of the tools that 1039 00:47:36,620 --> 00:47:40,000 exist at the moment general generative AI, we want 1040 00:47:40,000 --> 00:47:42,120 something that's been more specific. So, 1041 00:47:42,140 --> 00:47:44,500 yeah, the book's very much about like how we use 1042 00:47:44,500 --> 00:47:46,440 these tools. It's exploring things like prompt 1043 00:47:46,440 --> 00:47:49,260 engineering, fine tuning, our retrieval argumented 1044 00:47:49,260 --> 00:47:51,980 generation. But it's also as much about us 1045 00:47:51,980 --> 00:47:54,520 as individuals. How do we identify the places 1046 00:47:54,520 --> 00:47:56,440 that they can be useful? How 1047 00:47:56,440 --> 00:48:00,580 can we have like healthy level of skepticism, but 1048 00:48:00,580 --> 00:48:03,000 not be so skeptical that we become cynical about 1049 00:48:03,000 --> 00:48:06,120 these tools as well? And yeah, generally just sort 1050 00:48:06,120 --> 00:48:09,480 of work out ways in which we can use the general AI 1051 00:48:09,480 --> 00:48:14,020 tools to help us enhance our testing, you know, test 1052 00:48:14,020 --> 00:48:16,980 faster, test deeper, that sort of idea. But 1053 00:48:16,980 --> 00:48:19,160 certainly not replace us, that's for sure. At 1054 00:48:19,160 --> 00:48:22,560 least not now. Right, I think a lot of people 1055 00:48:22,560 --> 00:48:25,320 are thinking, you know, scarily that these kind of 1056 00:48:25,320 --> 00:48:27,900 tools will kind of like effectively eliminate their 1057 00:48:27,900 --> 00:48:29,800 jobs or their roles, right? But 1058 00:48:29,800 --> 00:48:32,240 I can also see some kind of evolution, right? Maybe 1059 00:48:32,240 --> 00:48:34,940 developers will be able to write more of these test 1060 00:48:34,940 --> 00:48:38,420 cases, right? Or maybe testers can also use 1061 00:48:38,420 --> 00:48:40,860 that to improve the code itself, not just reporting 1062 00:48:40,860 --> 00:48:44,380 bug and let developers fix those bugs, but also kind 1063 00:48:44,380 --> 00:48:47,100 of like help to fix it along the way. So 1064 00:48:47,100 --> 00:48:48,960 I think what kind of things that maybe after you 1065 00:48:48,960 --> 00:48:51,260 play around with this AI, what kind of things that 1066 00:48:51,260 --> 00:48:53,480 probably would change because of the introduction 1067 00:48:53,480 --> 00:48:56,680 of AI, not talking just about elimination of roles, 1068 00:48:56,880 --> 00:48:59,560 but I think, I don't know, like an evolution of skill 1069 00:48:59,560 --> 00:49:02,400 set or maybe a different approach of collaboration 1070 00:49:02,400 --> 00:49:05,680 altogether. Yeah, Gen AI stuff has been 1071 00:49:05,680 --> 00:49:08,040 around fairly recently, but like there's been a lot of 1072 00:49:08,040 --> 00:49:10,560 sort of AI work, machine learning work that's been 1073 00:49:10,560 --> 00:49:12,820 done in the past that we, especially in the testing 1074 00:49:12,820 --> 00:49:15,000 specs, we've seen around things like visual 1075 00:49:15,000 --> 00:49:17,720 comparison tools, for example. So, 1076 00:49:17,940 --> 00:49:19,440 you know, I get asked this question, like, you know, 1077 00:49:19,720 --> 00:49:21,780 well, AI take our testing jobs. And 1078 00:49:21,780 --> 00:49:23,460 I always want to say, like, if they take our 1079 00:49:23,460 --> 00:49:26,800 testing jobs, then we have bigger problems to deal 1080 00:49:26,800 --> 00:49:30,260 with than our jobs right now, because distinctly 1081 00:49:30,260 --> 00:49:34,700 testing and quality is a human driven, heuristic 1082 00:49:34,700 --> 00:49:38,000 driven sort of matter. But I agree with you, I think 1083 00:49:38,000 --> 00:49:41,540 that it has the impact to evolve our roles, I think 1084 00:49:41,540 --> 00:49:43,740 probably more on the developer side than on the 1085 00:49:43,740 --> 00:49:46,460 testing side, as it stands. I 1086 00:49:46,460 --> 00:49:49,260 think there's a lot of talk about how actually, as 1087 00:49:49,260 --> 00:49:53,720 we start to rely on tools like co -pilot as developers 1088 00:49:53,720 --> 00:49:57,500 to build our frameworks, then there's an argument 1089 00:49:57,500 --> 00:49:59,920 that it's because, you know, it's garbage in 1090 00:49:59,920 --> 00:50:01,860 garbage out. So if this thing is trained 1091 00:50:01,860 --> 00:50:04,400 on bad patterns, it's going to output bad patterns, 1092 00:50:04,580 --> 00:50:07,480 which is, you know, means boom time for testers, 1093 00:50:07,600 --> 00:50:10,040 because we're going to have more bugs to find and more 1094 00:50:10,040 --> 00:50:12,840 testing to do. If you kind of cynically look 1095 00:50:12,840 --> 00:50:14,780 at it from that sort of angle. So 1096 00:50:14,780 --> 00:50:17,460 there is that sort of challenge of how do you 1097 00:50:17,460 --> 00:50:21,240 test the product that has been developed by not just 1098 00:50:21,240 --> 00:50:24,500 an individual or collection of individuals, but also by 1099 00:50:24,500 --> 00:50:28,020 a highly probabilistic machine as well. So 1100 00:50:28,020 --> 00:50:29,940 there's that sort of factor. I 1101 00:50:29,940 --> 00:50:32,760 think that, yeah, the developers who are having 1102 00:50:32,760 --> 00:50:34,960 success with these tools, and I think it's going to 1103 00:50:34,960 --> 00:50:36,960 apply in the sort of testing spaces, are the 1104 00:50:36,960 --> 00:50:39,760 people who are using it for very specific tasks, as 1105 00:50:39,760 --> 00:50:43,520 I say. So I'm on a Slack group with 1106 00:50:43,520 --> 00:50:45,820 some developers who are working with things like co 1107 00:50:45,820 --> 00:50:47,800 -pilot. And what they've said, what's 1108 00:50:47,800 --> 00:50:50,120 really interesting is that they can quickly knock up 1109 00:50:50,120 --> 00:50:52,380 some code, knock up some tests, and then it gives 1110 00:50:52,380 --> 00:50:54,780 them more time to sort of play around and refactor 1111 00:50:54,780 --> 00:50:56,620 the code. So to give it that extra 1112 00:50:56,620 --> 00:50:58,660 polish. So there's almost like a 1113 00:50:58,660 --> 00:51:00,960 Pareto's law thing going on, where the tool gets 1114 00:51:00,960 --> 00:51:04,180 them 80 % of the way, and then that individual sort 1115 00:51:04,180 --> 00:51:06,240 of factor comes in there. I 1116 00:51:06,240 --> 00:51:08,200 think as well, there's going to be rolled like as 1117 00:51:08,200 --> 00:51:10,220 these tools become more and more prevalent, there's 1118 00:51:10,220 --> 00:51:12,560 going to be more interesting people who can 1119 00:51:12,560 --> 00:51:16,120 write prompts in people who can engineer these type of 1120 00:51:16,120 --> 00:51:18,300 tools as well. So not necessarily data 1121 00:51:18,300 --> 00:51:21,160 scientists and AI scientists, but can you 1122 00:51:21,160 --> 00:51:23,860 tune a model? Can you build a rack setup? 1123 00:51:24,540 --> 00:51:26,360 And conversely, you know, for 1124 00:51:26,360 --> 00:51:28,980 testers, can you test these systems? And 1125 00:51:28,980 --> 00:51:31,720 how do you deal with a deterministic system versus 1126 00:51:31,720 --> 00:51:33,620 an indeterministic system? We've 1127 00:51:33,620 --> 00:51:37,280 all been taught how to build test cases. You 1128 00:51:37,280 --> 00:51:39,300 can't do that in this context. So 1129 00:51:39,300 --> 00:51:41,960 there's new skills to kind of learn there. I 1130 00:51:41,960 --> 00:51:45,220 think on the not so great side is, and I think that 1131 00:51:45,220 --> 00:51:47,700 this applies to quite a lot of industries as well 1132 00:51:47,700 --> 00:51:50,040 as not just software development is, what does 1133 00:51:50,040 --> 00:51:52,780 this mean for people coming into the industry? You 1134 00:51:52,780 --> 00:51:55,840 know, it's all a good me talking from a perspective 1135 00:51:55,840 --> 00:51:59,400 of privilege of 20 years of experience working in 1136 00:51:59,400 --> 00:52:02,040 the quality space. But how does someone is just 1137 00:52:02,040 --> 00:52:03,940 coming into testing? How does someone is just coming 1138 00:52:03,940 --> 00:52:06,520 into development, learn those sort of important 1139 00:52:06,520 --> 00:52:08,800 skills? And where do those jobs come 1140 00:52:08,800 --> 00:52:10,820 from? Because, you know, if you've got 1141 00:52:10,820 --> 00:52:13,420 one large language model that's doing the work of 1142 00:52:13,420 --> 00:52:16,880 20 junior developers, that's probably going to mean that 1143 00:52:16,880 --> 00:52:18,860 they're not going to be hiring 19 of those in the 1144 00:52:18,860 --> 00:52:21,300 future. So I think there are questions 1145 00:52:21,300 --> 00:52:24,480 around that sort of aspect of moving into this 1146 00:52:24,480 --> 00:52:26,920 industry and how that will impact it. I 1147 00:52:26,920 --> 00:52:29,440 think, yeah, for a lot, it's going to be this 1148 00:52:29,440 --> 00:52:32,220 evolution of like, how do you build a relationship 1149 00:52:32,220 --> 00:52:36,580 with these tools in a way that utilizes both your 1150 00:52:36,580 --> 00:52:39,040 skills and skills and tools? Because 1151 00:52:39,040 --> 00:52:42,140 if you're not using them, you're not going to be as 1152 00:52:42,140 --> 00:52:44,360 performant as the person who is. And 1153 00:52:44,360 --> 00:52:46,640 if we rely too much on these types of tools, 1154 00:52:46,640 --> 00:52:50,320 they're just going to end up producing stuff that's 1155 00:52:50,320 --> 00:52:53,080 not very valuable to us, because they're not 1156 00:52:53,080 --> 00:52:54,760 designed, they're not chained to our context. 1157 00:52:55,360 --> 00:52:56,880 Yeah, you brought up a very 1158 00:52:56,880 --> 00:52:59,380 interesting point here for many of us to reflect, 1159 00:52:59,620 --> 00:53:01,380 right? So these LLM tools, the JAN 1160 00:53:01,380 --> 00:53:04,360 AI tool is currently still undeterministic, right? So 1161 00:53:04,360 --> 00:53:07,420 it's highly likely when you ask a question, it will 1162 00:53:07,420 --> 00:53:09,500 come up with a answer, then you ask again, it 1163 00:53:09,500 --> 00:53:11,420 will come up with b answers, like variant of 1164 00:53:11,420 --> 00:53:13,080 it. So I think it's very important 1165 00:53:13,080 --> 00:53:15,820 as a testers, your job is again to a certain quality. 1166 00:53:15,940 --> 00:53:17,380 So how can you, as a certain 1167 00:53:17,380 --> 00:53:19,500 quality, if you are undeterministic, so to 1168 00:53:19,500 --> 00:53:21,860 speak, right? So I think the job, the role 1169 00:53:21,860 --> 00:53:24,860 of testers most likely will not be eliminated. I 1170 00:53:24,860 --> 00:53:26,920 may be wrong. So maybe one day we can get 1171 00:53:26,920 --> 00:53:29,580 a much improved AI. But I think here the key is 1172 00:53:29,580 --> 00:53:31,720 to use AI as an assistant, right? It's 1173 00:53:31,720 --> 00:53:35,140 not to replace anyone and use that to boost our 1174 00:53:35,140 --> 00:53:37,340 productivity or kind of like to cover all these 1175 00:53:37,340 --> 00:53:40,320 boilerplates, like generating data or kind of like 1176 00:53:40,320 --> 00:53:42,500 starting the code itself. So 1177 00:53:42,508 --> 00:53:44,340 I think be more optimistic. And 1178 00:53:44,340 --> 00:53:46,440 I think we are looking forward for the book. I 1179 00:53:46,440 --> 00:53:48,840 hope to probably also read that book. And 1180 00:53:48,840 --> 00:53:50,720 if we can also cover another episode for that. 1181 00:53:51,080 --> 00:53:52,200 So thanks again, Mark, for 1182 00:53:52,200 --> 00:53:55,100 this interesting conversation about testing. Unfortunately, 1183 00:53:55,380 --> 00:53:57,760 we have to wrap up. But before I let you go, I 1184 00:53:57,760 --> 00:54:00,880 have one last question. I always ask my guests, which 1185 00:54:00,880 --> 00:54:03,100 I call the three technical leadership wisdom. So 1186 00:54:03,100 --> 00:54:05,260 if you can think of it just like an advice that 1187 00:54:05,260 --> 00:54:07,280 you want to give to us, maybe if you can share 1188 00:54:07,280 --> 00:54:10,100 your version for us to learn from you. Three 1189 00:54:10,100 --> 00:54:12,480 things to think about. So I think obviously the first 1190 00:54:12,480 --> 00:54:15,040 one's going to be context is key given that I've 1191 00:54:15,040 --> 00:54:18,340 banged on about that for nearly 600 pages across two 1192 00:54:18,340 --> 00:54:20,580 books. I think that is a truly 1193 00:54:20,580 --> 00:54:24,220 important aspect is that there is no one size fits 1194 00:54:24,220 --> 00:54:26,820 all solution to everything. I 1195 00:54:26,820 --> 00:54:29,160 think as well, you know, from the perspective of 1196 00:54:29,160 --> 00:54:32,360 being a tester, from being a quality engineer, I'm 1197 00:54:32,360 --> 00:54:34,640 still learning this myself, but being sort of a 1198 00:54:34,640 --> 00:54:37,620 servant leader is massively important as well. So 1199 00:54:37,620 --> 00:54:41,360 responding to what's going on around you and helping 1200 00:54:41,360 --> 00:54:44,140 people with problems is going to be much more 1201 00:54:44,140 --> 00:54:47,820 productive than kind of applying your own will for 1202 00:54:47,820 --> 00:54:50,880 your own perspective on things as well. And 1203 00:54:50,880 --> 00:54:52,700 as well, like I think we're talking about the AI 1204 00:54:52,700 --> 00:54:55,660 stuff as well is reflecting on yourself, reflecting on 1205 00:54:55,660 --> 00:54:59,180 your abilities. And so I'm a big advocate of 1206 00:54:59,180 --> 00:55:02,160 continuous learning and rules to read new stuff, 1207 00:55:02,640 --> 00:55:05,780 expose myself to new things as well, not just stuff in 1208 00:55:05,780 --> 00:55:09,040 the software development space, but in the arts as 1209 00:55:09,040 --> 00:55:12,180 well, I learned loads from in terms of teaching and 1210 00:55:12,180 --> 00:55:14,480 presenting from stand up comedy. I've 1211 00:55:14,480 --> 00:55:16,860 never gone up on the stage and done a set or anything 1212 00:55:16,860 --> 00:55:18,500 like that, but there's a lot we can learn from 1213 00:55:18,500 --> 00:55:22,080 people as well and draw those skillsets in as well. 1214 00:55:22,600 --> 00:55:23,280 So yeah, that would be my 1215 00:55:23,280 --> 00:55:25,940 three. Wow, it's very interesting. So 1216 00:55:25,948 --> 00:55:27,640 first is about context, right? Again, 1217 00:55:27,820 --> 00:55:30,720 like no one thing fits all the kind of a problem, 1218 00:55:30,880 --> 00:55:32,640 right? So I think I also like the 1219 00:55:32,640 --> 00:55:34,880 last part where you kind of like learn from 1220 00:55:34,880 --> 00:55:37,620 different various industries or different kind of 1221 00:55:37,620 --> 00:55:40,120 philosophies or comedy and things like that. And 1222 00:55:40,120 --> 00:55:42,100 you can infuse that to your role. I 1223 00:55:42,100 --> 00:55:44,000 think that's really powerful as well. So 1224 00:55:44,000 --> 00:55:46,780 Mark, if people would love to connect with you, ask 1225 00:55:46,780 --> 00:55:49,180 you more questions, maybe is there a place where 1226 00:55:49,180 --> 00:55:51,260 they can reach you online? So 1227 00:55:51,260 --> 00:55:54,400 usually hovering around on LinkedIn, so you find me 1228 00:55:54,400 --> 00:55:56,320 as Mark Winteringham there. I 1229 00:55:56,320 --> 00:55:59,600 am also still on Twitter slash X, what's left of 1230 00:55:59,600 --> 00:56:01,700 it. That's a two bit tester with a 1231 00:56:01,700 --> 00:56:03,900 number two. And also have my website, 1232 00:56:04,160 --> 00:56:08,340 mwtestconsultancy .co .uk, which I hope to be adding 1233 00:56:08,340 --> 00:56:11,580 some more stuff to once I finish the second book. 1234 00:56:12,100 --> 00:56:14,180 Right. I highly recommend your 1235 00:56:14,180 --> 00:56:18,020 testing web API book. So it's not just testing web 1236 00:56:18,020 --> 00:56:20,740 APIs guys. So it's also holistic approach, 1237 00:56:20,900 --> 00:56:22,620 right? Starting from identifying quality 1238 00:56:22,620 --> 00:56:25,020 characteristics, race approach and things like 1239 00:56:25,020 --> 00:56:26,220 that. I think it's kind of like 1240 00:56:26,220 --> 00:56:28,300 comprehensive. And I'm really looking forward 1241 00:56:28,300 --> 00:56:30,820 for the second book, AI assisted testing. I 1242 00:56:30,820 --> 00:56:33,800 think everyone is excited about potential AI. And 1243 00:56:33,800 --> 00:56:35,640 I think we are looking forward for some tips from 1244 00:56:35,640 --> 00:56:37,460 you on how to use it more properly. So 1245 00:56:37,460 --> 00:56:39,960 good luck with your process of writing the second book. 1246 00:56:40,460 --> 00:56:41,100 Thank you very much.