1 00:00:00,040 --> 00:00:01,720 Don't do things just because auditors tell you. 2 00:00:01,720 --> 00:00:03,200 If you don't understand why we're telling you to do 3 00:00:03,200 --> 00:00:06,480 something, challenge us because you should never do something 4 00:00:06,520 --> 00:00:08,480 just because the auditors want you to do it. 5 00:00:08,880 --> 00:00:11,320 The auditors should be able to explain we want you to do this 6 00:00:11,320 --> 00:00:14,840 because here's the risk, and you need to control that risk in 7 00:00:14,880 --> 00:00:17,600 accordance with your risk appetite intolerance. 8 00:00:22,520 --> 00:00:25,680 Hey everyone, my name is Henry Surya Viravan. 9 00:00:27,520 --> 00:00:30,800 And you're listening to the Technical Journal Podcast, the 10 00:00:30,800 --> 00:00:33,480 show where I'll be bringing you, the greatest technical leaders, 11 00:00:33,760 --> 00:00:37,360 practitioners and thought leaders in the industry to 12 00:00:37,360 --> 00:00:41,600 discuss about their journey, ideas and practices that we all 13 00:00:41,600 --> 00:00:45,120 can learn and apply to build a highly performing technical team 14 00:00:45,640 --> 00:00:47,840 and to make an impact in your personal work. 15 00:00:48,480 --> 00:00:56,440 So let's dive into our journal. Hello again everyone. 16 00:00:56,740 --> 00:00:59,540 You're listening to the Technijunal Podcast, the podcast 17 00:00:59,540 --> 00:01:01,340 where you can learn about technical leadership and 18 00:01:01,340 --> 00:01:04,660 excellence from my conversations with great thought leaders in 19 00:01:04,660 --> 00:01:07,460 the tech industry. If you haven't, please follow 20 00:01:07,460 --> 00:01:10,700 the show on your podcast app and social media on LinkedIn, 21 00:01:10,700 --> 00:01:14,700 Twitter and Instagram, and also video contents on YouTube and 22 00:01:14,700 --> 00:01:18,300 TikTok to support my work in producing this podcast and its 23 00:01:18,300 --> 00:01:20,980 various contents. You can buy me a coffee at 24 00:01:20,980 --> 00:01:24,710 technijunal.net tip. Or subscribe as a patron at 25 00:01:24,710 --> 00:01:27,110 Technically. Journal dot dev slash patron. 26 00:01:28,350 --> 00:01:31,070 My guest for today's episode is Clarissa Lucas. 27 00:01:31,470 --> 00:01:34,870 Clarissa is an audit and risk management leader and the author 28 00:01:34,870 --> 00:01:39,030 of Beyond Agile Auditing. In this episode, Clarissa shared 29 00:01:39,070 --> 00:01:42,390 a novel approach to internal auditing called Auditing with 30 00:01:42,390 --> 00:01:44,990 Agility. She shared this concept at the 31 00:01:44,990 --> 00:01:49,430 DevOps Enterprise Summit 2022, which drew some parallels to the 32 00:01:49,430 --> 00:01:51,870 revolutionary birth of the DevOps movement. 33 00:01:52,570 --> 00:01:55,730 Clarissa explained the three core components of auditing with 34 00:01:55,730 --> 00:01:59,970 agility, which are Value Driven Auditing, Integrated Auditing 35 00:01:59,970 --> 00:02:04,610 2.0, and Adaptable Auditing. I hope you enjoy listening to 36 00:02:04,610 --> 00:02:07,930 this episode and learning a new approach to internal auditing 37 00:02:07,970 --> 00:02:10,810 that doesn't cause you to drag working with your auditors. 38 00:02:11,210 --> 00:02:14,850 From my experience, I sincerely believe we need to revolutionize 39 00:02:14,850 --> 00:02:18,010 the way auditing is done in order to bring a better value 40 00:02:18,010 --> 00:02:21,570 for the organization and make the experience better and more 41 00:02:21,570 --> 00:02:24,310 productive. If you like this episode, it 42 00:02:24,310 --> 00:02:27,110 will be really great if you can help me share this with your 43 00:02:27,110 --> 00:02:30,350 colleagues, your friends and communities and leave a 5 star 44 00:02:30,350 --> 00:02:33,110 rating and review on Apple Podcasts and Spotify. 45 00:02:33,670 --> 00:02:36,830 It will help me a lot in getting more people discover and listen 46 00:02:36,830 --> 00:02:40,110 to this podcast. Let's go to my conversation with 47 00:02:40,110 --> 00:02:43,070 Clarissa after quick words from our sponsor. 48 00:02:43,630 --> 00:02:45,390 Are you looking for a new cool swag? 49 00:02:45,830 --> 00:02:49,190 Technically, Juno now offers you some Swags that you can purchase 50 00:02:49,190 --> 00:02:51,730 online. These swags are printed on 51 00:02:51,730 --> 00:02:55,290 demand based on your preference and will be delivered safely to 52 00:02:55,290 --> 00:02:58,170 you all over the world. Where shipping is available, 53 00:02:58,690 --> 00:03:01,650 Check out all the cool swags available by visiting Tech Lead. 54 00:03:01,650 --> 00:03:05,130 Journal dot dev slash shop And don't forget to break yourself 55 00:03:05,250 --> 00:03:07,370 once you receive any of those swags. 56 00:03:10,130 --> 00:03:12,450 Everyone, welcome back to another new episode of the Tech 57 00:03:12,450 --> 00:03:15,250 Lead Journal Podcast. Today I have with me Clarissa 58 00:03:15,250 --> 00:03:17,290 Lucas. She's the author of a book 59 00:03:17,290 --> 00:03:21,200 titled Beyond HR Auditing. The three core components to 60 00:03:21,200 --> 00:03:23,720 revolutionize your internal audit practices. 61 00:03:24,040 --> 00:03:26,320 As you can tell from the title, we are going to talk about 62 00:03:26,320 --> 00:03:29,080 auditing. I myself have to be honest, I'm 63 00:03:29,080 --> 00:03:31,240 not the person who likes to be audited. 64 00:03:31,480 --> 00:03:34,400 No, I know a lot about auditing. So this episode I think is going 65 00:03:34,400 --> 00:03:35,840 to be insightful for me at least. 66 00:03:36,120 --> 00:03:39,040 And I hope it will also give you some learning experience about 67 00:03:39,040 --> 00:03:41,920 how we can do audit better. So, Clarissa, thank you so much 68 00:03:41,920 --> 00:03:43,840 for this time. I'm really looking forward to 69 00:03:43,840 --> 00:03:46,920 learn from you about auditing. Henry, thanks for having me. 70 00:03:46,920 --> 00:03:50,200 And you are not alone. And not really being excited 71 00:03:50,200 --> 00:03:53,240 about getting audited. That was a big reason why I 72 00:03:53,240 --> 00:03:55,880 wrote the book. So I'll introduce myself. 73 00:03:55,880 --> 00:03:58,800 But I do want to dive into that a little bit, this part of about 74 00:03:58,800 --> 00:04:01,960 me so. One of my personality traits is 75 00:04:01,960 --> 00:04:04,640 I love when people get along and I struggle when they don't get 76 00:04:04,640 --> 00:04:07,120 along. So when people don't like the 77 00:04:07,120 --> 00:04:10,600 auditors to be there, they see me as the bad guy and I get it. 78 00:04:10,680 --> 00:04:13,080 But those are things that I want to fix. 79 00:04:13,160 --> 00:04:17,200 So you're not alone in that. I am trying one organization, 80 00:04:17,200 --> 00:04:20,720 one person at a time to turn that adversarial relationship, 81 00:04:20,720 --> 00:04:23,520 turn that fear of the auditors into something that's super 82 00:04:23,520 --> 00:04:26,280 valuable. So I am so happy that you have 83 00:04:26,280 --> 00:04:30,260 me on the show today. I have spent most of my career 84 00:04:30,260 --> 00:04:33,180 in internal audit or risk management second line function, 85 00:04:33,180 --> 00:04:36,380 so maybe not always as an internal auditor, but usually in 86 00:04:36,380 --> 00:04:38,220 that type of role where somebody's coming and they feel 87 00:04:38,220 --> 00:04:41,300 like they're being audited has to do speaking engagement on 88 00:04:41,300 --> 00:04:43,420 this topic. This is something that is super 89 00:04:43,420 --> 00:04:46,140 near and dear to my heart and as you mentioned, I'm a published 90 00:04:46,140 --> 00:04:48,220 author. My book Beyond Agile Auditing 91 00:04:48,220 --> 00:04:49,740 just came out a couple of weeks ago. 92 00:04:49,740 --> 00:04:51,860 So this has been a whole new learning experience. 93 00:04:52,340 --> 00:04:55,220 A few major highlights in my career. 94 00:04:55,610 --> 00:04:59,050 Are presenting at my first DevOps Enterprise Summit, taking 95 00:04:59,050 --> 00:05:01,890 on my current leadership role where I pivoted my focus from 96 00:05:01,890 --> 00:05:04,490 individual accomplishments to people, and then publishing my 97 00:05:04,490 --> 00:05:05,850 book. If you don't mind, I'm going to 98 00:05:05,850 --> 00:05:08,370 take a couple minutes and talk through each of those because 99 00:05:08,370 --> 00:05:11,010 they think that'll help paint the picture for the rest of the 100 00:05:11,010 --> 00:05:14,090 episode today. So first one DevOps Enterprise 101 00:05:14,090 --> 00:05:17,890 Summit in 2020. I wasn't too far into my current 102 00:05:17,890 --> 00:05:21,570 role and I had the opportunity to speak at the DevOps 103 00:05:21,570 --> 00:05:24,610 Enterprise Summit, so I was new to technology auditing. 104 00:05:24,970 --> 00:05:28,210 Most of my career in auditing had been on the operational 105 00:05:28,210 --> 00:05:31,010 side, not necessarily on the technology side, but this was a 106 00:05:31,010 --> 00:05:33,250 new adventure for me. I love learning things and 107 00:05:33,250 --> 00:05:36,450 technology is really important, so I was intrigued by taking on 108 00:05:36,450 --> 00:05:38,970 that role. It was virtual that year, this 109 00:05:38,970 --> 00:05:40,730 was 2020, the start of the pandemic. 110 00:05:41,010 --> 00:05:43,450 Public speaking has always been a source of anxiety for me, even 111 00:05:43,450 --> 00:05:45,250 though that's a lot of what I do now. 112 00:05:45,570 --> 00:05:51,290 Learning and growth is important and so public speaking and I'm 113 00:05:51,290 --> 00:05:53,810 the only or one of the only auditors at this. 114 00:05:54,150 --> 00:05:58,230 Conference that is focused on technology leaders, really smart 115 00:05:58,230 --> 00:06:01,390 technology people, and I didn't have that background either. 116 00:06:01,670 --> 00:06:05,670 To say that it was overwhelming and terrifying for me was 117 00:06:05,670 --> 00:06:08,510 probably an understatement. So that virtual environment 118 00:06:08,510 --> 00:06:11,670 since the pandemic was there, made that a really great 119 00:06:11,670 --> 00:06:14,590 stepping stone for me. It made it super enjoyable. 120 00:06:14,590 --> 00:06:17,910 It helped me build that confidence, which has been a 121 00:06:17,910 --> 00:06:19,950 stepping stone for a lot of these other types of 122 00:06:19,950 --> 00:06:23,310 opportunities. Another reason that that was 123 00:06:23,310 --> 00:06:26,950 such a pivotal moment for me was while I was there and presenting 124 00:06:27,150 --> 00:06:29,670 a lot of the questions that were coming through from the audience 125 00:06:30,030 --> 00:06:34,510 really opened my eyes to a number of misconceptions about 126 00:06:34,510 --> 00:06:37,350 auditors that led to that fear. And not looking forward to the 127 00:06:37,350 --> 00:06:41,630 auditors coming and seeing auditors as roadblocks and 128 00:06:41,630 --> 00:06:44,750 seeing them as getting in the way of technology organizations 129 00:06:44,750 --> 00:06:48,630 progressing in better ways of working in things like DevOps. 130 00:06:49,110 --> 00:06:52,670 So it really started my journey to I need to tear down these 131 00:06:52,670 --> 00:06:56,910 silos and help bring some truth to these misconceptions and help 132 00:06:56,910 --> 00:06:58,510 these two groups get along better. 133 00:06:58,510 --> 00:07:01,870 Because there was a huge opportunity to have them 134 00:07:01,870 --> 00:07:04,950 leverage each other instead of getting in each other's ways. 135 00:07:04,990 --> 00:07:08,150 Words are hard today, so that was the first turning point. 136 00:07:08,550 --> 00:07:11,030 Another one was when I took on this role, so we're backing up. 137 00:07:11,070 --> 00:07:14,590 The first one was 2020, this is in 2018-2019. 138 00:07:14,830 --> 00:07:16,910 I took on this role as a technology audit leader. 139 00:07:17,330 --> 00:07:20,130 And I'd had leadership positions before, but I still hadn't 140 00:07:20,130 --> 00:07:23,330 really mastered that transition from individual contributor and 141 00:07:23,330 --> 00:07:27,290 focusing on getting the things done to being a leader and 142 00:07:27,290 --> 00:07:30,970 focusing on the people. And that shift was super pivotal 143 00:07:30,970 --> 00:07:34,690 for me as well and just really helped me become a better leader 144 00:07:34,930 --> 00:07:39,370 both to my direct reports and in the audit role where I'm leading 145 00:07:39,370 --> 00:07:42,330 conversations and leading activities where there are 146 00:07:42,330 --> 00:07:44,890 multiple people in the room and you really have to focus on 147 00:07:45,050 --> 00:07:48,320 people more so than. The mechanics of getting things 148 00:07:48,320 --> 00:07:51,040 done. So those two led to my third, 149 00:07:51,440 --> 00:07:53,200 which was publishing my first book. 150 00:07:53,720 --> 00:07:56,840 Had I not experienced both of those earlier turning points and 151 00:07:56,840 --> 00:07:59,520 experiences, I definitely would not be here today. 152 00:07:59,920 --> 00:08:03,680 The experience of publishing a book and taking on more and more 153 00:08:03,680 --> 00:08:05,800 speaking engagements and connecting with people that I 154 00:08:05,800 --> 00:08:09,240 normally wouldn't have had the opportunity to has been an 155 00:08:09,440 --> 00:08:12,680 absolutely incredible experience, absolutely 156 00:08:12,680 --> 00:08:16,250 one-of-a-kind. I love helping people, and this 157 00:08:16,250 --> 00:08:19,050 book has been a great accelerator to enable me to 158 00:08:19,130 --> 00:08:21,410 connect with people and start helping people and like I said, 159 00:08:21,450 --> 00:08:24,090 tearing down those silos and shining a light on those 160 00:08:24,090 --> 00:08:26,810 misconceptions. Wow, thank you for sharing your 161 00:08:26,810 --> 00:08:28,610 story. I think that's really great. 162 00:08:28,690 --> 00:08:32,530 So I myself, I'm pretty amazed that you got quite a good 163 00:08:32,530 --> 00:08:34,929 reception in the DevOps Enterprise Summit talking about 164 00:08:34,929 --> 00:08:36,370 audit. So there must be something. 165 00:08:36,610 --> 00:08:39,250 Maybe we'll talk about that later as well, but for people 166 00:08:39,250 --> 00:08:42,890 who are new to getting to know in depth about auditing. 167 00:08:43,240 --> 00:08:44,720 Maybe we can start from there. First, right? 168 00:08:44,800 --> 00:08:46,920 What is actually the real purpose of auditing? 169 00:08:46,920 --> 00:08:48,960 What is auditing and internal auditing? 170 00:08:48,960 --> 00:08:51,800 Specifically, if you mentioned in the title right, is there 171 00:08:51,800 --> 00:08:54,800 anything that you can enlighten us about auditing here? 172 00:08:55,400 --> 00:08:57,360 Yep. So a lot of people might think, 173 00:08:57,480 --> 00:09:00,000 OK, the purpose of auditing is to shine a light on things that 174 00:09:00,000 --> 00:09:01,680 are going wrong and make you look bad. 175 00:09:01,920 --> 00:09:05,000 I can assure you that is not what we're here to do. 176 00:09:05,360 --> 00:09:07,920 And the really cool thing about internal auditors is we work for 177 00:09:07,920 --> 00:09:10,360 the same organization that the people we're auditing do. 178 00:09:10,400 --> 00:09:12,760 So we are. Different from an external 179 00:09:12,760 --> 00:09:16,600 auditor, we are different from other internal assurance 180 00:09:16,600 --> 00:09:19,680 functions because we do have a bit of that step back, that 181 00:09:19,680 --> 00:09:21,760 independence. But we are still part of the 182 00:09:21,760 --> 00:09:24,400 same organization. So we are on the same team. 183 00:09:24,400 --> 00:09:26,560 And I know that sounds like, Oh yeah, yeah, you're you're all on 184 00:09:26,560 --> 00:09:28,960 the same team, I promise you we are. 185 00:09:29,360 --> 00:09:32,240 So the purpose of internal audit is to be independent and 186 00:09:32,240 --> 00:09:35,360 objective. We try not to be as biased by, 187 00:09:35,360 --> 00:09:38,000 you know, if you're in the weeds every day doing this. 188 00:09:38,330 --> 00:09:39,210 Of course, you're doing it great. 189 00:09:39,210 --> 00:09:41,570 Like it's wonderful and I'm sure it is. 190 00:09:41,570 --> 00:09:45,090 But there's a value that that objective perspective, that 191 00:09:45,090 --> 00:09:47,570 fresh perspective can bring to those things. 192 00:09:47,570 --> 00:09:50,170 So our goal is to add value to our organizations. 193 00:09:50,330 --> 00:09:52,770 If I had to summarize it, it's to add value. 194 00:09:53,010 --> 00:09:56,330 And we really want to do that through partnering with our 195 00:09:56,330 --> 00:09:59,130 clients and bringing that fresh perspective and providing our 196 00:09:59,130 --> 00:10:01,530 clients with value through assurance. 197 00:10:01,530 --> 00:10:05,850 So letting them know the things that you rely on to go right, 198 00:10:06,250 --> 00:10:08,240 are they going to go right? Is there a good chance that 199 00:10:08,240 --> 00:10:10,240 they're going to go right? Or is something not working the 200 00:10:10,240 --> 00:10:12,160 way you think it's going to work and you're probably going to run 201 00:10:12,160 --> 00:10:16,680 into problems down the road? Or do you have the mechanisms in 202 00:10:16,680 --> 00:10:19,320 place to make sure that when it doesn't go right, you're going 203 00:10:19,320 --> 00:10:21,480 to identify that in a timely manner and be able to fix it 204 00:10:21,480 --> 00:10:23,560 right away so that you can achieve your objectives? 205 00:10:23,760 --> 00:10:27,320 That's really why we're here. None of that is to make it look 206 00:10:27,320 --> 00:10:30,760 bad or to ruin your day or anything like that, which is 207 00:10:30,840 --> 00:10:33,360 probably what some people may have experienced, unfortunately. 208 00:10:34,160 --> 00:10:36,120 Right. I like when you say that you 209 00:10:36,120 --> 00:10:38,720 bring value as well to the organization, Of course, we are 210 00:10:38,720 --> 00:10:39,920 talking about internal auditors here. 211 00:10:39,960 --> 00:10:42,280 Yeah, external might be different, but it's an internal 212 00:10:42,280 --> 00:10:44,080 auditor. You also work together, right, 213 00:10:44,080 --> 00:10:46,800 to bring value. And I like specifically in the 214 00:10:46,800 --> 00:10:50,040 book you mentioned that internal auditors are much better or 215 00:10:50,040 --> 00:10:52,880 maybe you call them experts in risk and control, right. 216 00:10:52,880 --> 00:10:55,480 So things when you said when things go wrong, what mechanism 217 00:10:55,480 --> 00:10:58,960 we should have in place or how to make sure that things do 218 00:10:58,960 --> 00:11:01,360 actually go right. So I think that's also important 219 00:11:01,640 --> 00:11:04,800 when we said that many people. Dread being audited. 220 00:11:04,880 --> 00:11:07,440 There must be a reasons, definitely right? 221 00:11:07,520 --> 00:11:11,000 I myself maybe can share some of my frustration, but maybe from 222 00:11:11,000 --> 00:11:12,960 your point of view, First, what are some of the common 223 00:11:12,960 --> 00:11:15,360 challenges? Why there's a bad perception or 224 00:11:15,360 --> 00:11:17,440 maybe misconception about auditors? 225 00:11:18,400 --> 00:11:20,640 Yeah, I think when things go wrong, people are always looking 226 00:11:20,640 --> 00:11:21,840 at where we're the auditors here. 227 00:11:21,840 --> 00:11:24,360 So you know that sometimes would put the auditors on the 228 00:11:24,360 --> 00:11:28,520 defensive of we have to look at everything so that we don't get 229 00:11:28,520 --> 00:11:31,160 those fingers pointed at us, I also think. 230 00:11:31,500 --> 00:11:36,420 It's gotten potentially worse in the past few decades because we 231 00:11:36,420 --> 00:11:39,140 used to show up with checklists and here's what we're going to 232 00:11:39,140 --> 00:11:40,940 audit. And things didn't change very 233 00:11:40,940 --> 00:11:42,820 often. So a checklist that you dust off 234 00:11:42,820 --> 00:11:45,500 every year and do the same testing was effective for those 235 00:11:45,500 --> 00:11:47,900 purposes. But that is absolutely not the 236 00:11:47,900 --> 00:11:51,620 world we're living in today. Things change so quickly, so 237 00:11:51,620 --> 00:11:54,340 when auditors show up with that checklist and do the same thing 238 00:11:54,340 --> 00:11:56,140 that they did last time they were there. 239 00:11:56,550 --> 00:11:59,950 Clients are like, this is not helpful, like that checklist is 240 00:11:59,950 --> 00:12:03,150 so outdated and they're not digging into what's really 241 00:12:03,150 --> 00:12:05,710 important to me or maybe they are, maybe the checklist is 242 00:12:05,710 --> 00:12:08,470 still focused on those areas, but the auditors might have 243 00:12:08,470 --> 00:12:10,470 their heads down and are just focused on executing that 244 00:12:10,470 --> 00:12:12,270 checklist. And like when I mentioned moving 245 00:12:12,270 --> 00:12:15,150 from that individual contributor to this role, like focusing on 246 00:12:15,150 --> 00:12:17,950 executing versus understanding people and understanding their 247 00:12:17,950 --> 00:12:20,710 processes and what's important to them, we needed to make that 248 00:12:20,710 --> 00:12:22,690 shift so. I think you know with the rest 249 00:12:22,690 --> 00:12:25,490 of the organization keeping up with the pace of change and 250 00:12:25,490 --> 00:12:27,610 modernizing their ways of working, modernizing their 251 00:12:27,610 --> 00:12:31,010 technology and their processes as well and audit kind of got 252 00:12:31,010 --> 00:12:32,370 left in the dust for a little bit. 253 00:12:32,730 --> 00:12:36,490 And that I think also created some of those challenges and 254 00:12:36,490 --> 00:12:38,990 barriers and then. Yeah, Somebody's just going to 255 00:12:38,990 --> 00:12:41,510 show up and throw some unplanned work on your plate. 256 00:12:41,510 --> 00:12:43,070 That's not going to add any value. 257 00:12:43,070 --> 00:12:45,390 I don't blame you for not being thrilled that they're there. 258 00:12:45,390 --> 00:12:47,310 I mean, if somebody walked in here today and is like, do all 259 00:12:47,310 --> 00:12:49,110 this work, that's not going to help you at all. 260 00:12:49,110 --> 00:12:50,870 And you still have to get your other stuff done. 261 00:12:50,990 --> 00:12:53,710 I wouldn't be thrilled either. I'd be fearing or, you know, 262 00:12:53,710 --> 00:12:57,270 dreading like I think you said some of that person showing up. 263 00:12:57,270 --> 00:13:01,230 So I think those are some of the things that have led to that 264 00:13:01,230 --> 00:13:03,670 strained relationship, I'll say. Right. 265 00:13:04,160 --> 00:13:06,720 So when I read the, I think the first few chapters in your book, 266 00:13:06,720 --> 00:13:09,280 you mentioned also common challenges that you frequently 267 00:13:09,280 --> 00:13:12,320 find from either your previous organizations or from your 268 00:13:12,320 --> 00:13:15,040 customers clients. So I think when I read that some 269 00:13:15,040 --> 00:13:16,640 of them actually ring true to me. 270 00:13:16,680 --> 00:13:20,400 So for example, the things about us versus them, the silos, I 271 00:13:20,400 --> 00:13:22,760 think that's the first impression that I got as well, 272 00:13:22,760 --> 00:13:26,120 especially if the auditors do not come from the same team, 273 00:13:26,120 --> 00:13:27,480 right. They are just separate maybe 274 00:13:27,480 --> 00:13:30,400 reporting to different boss and they will just throw you 275 00:13:30,400 --> 00:13:33,280 checklist, OK, we're gonna do an audit for your system or 276 00:13:33,280 --> 00:13:36,090 whatever. And yeah, you have to just come 277 00:13:36,370 --> 00:13:38,450 prepared whenever there's any findings. 278 00:13:38,770 --> 00:13:41,730 So that is always not good. Because the first interaction 279 00:13:41,730 --> 00:13:44,130 itself is kind of like maybe many tensions, right? 280 00:13:44,130 --> 00:13:47,490 It's like, yes, you're policing us and we are like criminals. 281 00:13:48,010 --> 00:13:48,850 Yep. Yep. 282 00:13:48,850 --> 00:13:51,770 And that's not our intent, although I get that the way 283 00:13:51,770 --> 00:13:53,850 things have been working in the past, it feels like that, 284 00:13:53,850 --> 00:13:56,410 especially when you know, you mentioned they send the 285 00:13:56,410 --> 00:13:58,890 checklist to you, We send sometimes and you'll get this 286 00:13:58,890 --> 00:14:01,980 with external auditors as well. Here's our request list. 287 00:14:01,980 --> 00:14:04,260 So we're figuring out what we want to audit. 288 00:14:04,300 --> 00:14:07,140 We talk to you a little bit, figure out what it is you do. 289 00:14:07,140 --> 00:14:10,220 We sit over at our desks and we create our scope for our audit. 290 00:14:10,220 --> 00:14:13,380 We fill out a request list and we toss that over to our clients 291 00:14:13,780 --> 00:14:15,460 and it's usually written in audit terms. 292 00:14:15,460 --> 00:14:18,300 So you mentioned we're the experts in risks and controls. 293 00:14:18,300 --> 00:14:22,580 We speak in risks and controls. Most people outside of audit or 294 00:14:22,580 --> 00:14:25,860 risk functions do not speak in risks and controls. 295 00:14:26,230 --> 00:14:30,950 So it's typically in a different type of wording that then our 296 00:14:30,950 --> 00:14:33,390 clients are used to and they're stuck trying to figure out what 297 00:14:33,390 --> 00:14:36,710 the heck are these auditors actually looking for or even if 298 00:14:36,710 --> 00:14:39,630 it is clear what we're looking for, it might not actually be 299 00:14:39,630 --> 00:14:42,670 the documentation of the evidence that we need to test 300 00:14:42,670 --> 00:14:45,510 what we're looking at. So those silos really get in the 301 00:14:45,510 --> 00:14:50,350 way of a common understanding and really an opportunity to add 302 00:14:50,350 --> 00:14:52,470 value more efficiently. So I know you were going in a 303 00:14:52,470 --> 00:14:54,430 different direction, but I did want to point that out. 304 00:14:55,240 --> 00:14:58,240 Yeah, no problem. I would also love to share some 305 00:14:58,240 --> 00:15:01,240 of my point of view, right, the frustrations that I have so that 306 00:15:01,240 --> 00:15:03,760 we can discuss and maybe other people can relate as well. 307 00:15:04,040 --> 00:15:07,480 The other frustration point that I have is about for example, 308 00:15:07,480 --> 00:15:10,080 right, they give us some findings, but they don't seem to 309 00:15:10,080 --> 00:15:13,320 relate so much with the context that we are working in or maybe 310 00:15:13,320 --> 00:15:16,560 that comes from an outdated version of some documents like 311 00:15:16,560 --> 00:15:19,600 you mentioned because some of these comes from compliance 312 00:15:19,600 --> 00:15:23,270 which are probably. Created some years ago and they 313 00:15:23,270 --> 00:15:26,430 may not relate, but they create that as a finding and you just 314 00:15:26,430 --> 00:15:28,910 have to build some kind of rationale why this is not 315 00:15:28,910 --> 00:15:31,150 applicable for us before they can say, OK, check. 316 00:15:31,460 --> 00:15:34,660 And sometimes it goes through a few rounds of, you know, back 317 00:15:34,660 --> 00:15:36,220 and forth before they can accept that. 318 00:15:36,620 --> 00:15:38,860 Yeah. So some of the, I think that's 319 00:15:38,860 --> 00:15:43,500 inherent in those silos that you mentioned and not working as 320 00:15:43,500 --> 00:15:46,300 collaboratively together and not getting that based understanding 321 00:15:46,300 --> 00:15:49,100 of what is very important, what you mentioned from a compliance 322 00:15:49,100 --> 00:15:51,620 perspective, what are the current requirements. 323 00:15:52,040 --> 00:15:54,920 What are the most important compliance requirements today? 324 00:15:54,920 --> 00:15:57,840 Because there are so many different requirements, but what 325 00:15:57,840 --> 00:16:00,200 are the ones that are really impactful to you and your 326 00:16:00,200 --> 00:16:03,000 organization, both from a regulatory perspective, from an 327 00:16:03,000 --> 00:16:06,640 internal policies perspective? Because you're right, We could 328 00:16:06,640 --> 00:16:09,040 spend all this time over in this space to the left. 329 00:16:09,400 --> 00:16:12,880 But if that's not what's most important, if that's not working 330 00:16:12,880 --> 00:16:15,480 and we hand you a report that says these things are broken or 331 00:16:15,480 --> 00:16:18,040 you're not complying with these areas, you don't care. 332 00:16:18,040 --> 00:16:19,880 That was a waste of your time and my time. 333 00:16:20,440 --> 00:16:23,480 So what I talk about in the book generally is called auditing 334 00:16:23,480 --> 00:16:27,640 with Agility and it's a flexible approach where we break down 335 00:16:27,640 --> 00:16:31,080 those silos and we really focus on value. 336 00:16:31,080 --> 00:16:32,480 So there's three core components. 337 00:16:32,480 --> 00:16:36,280 The first one is value driven auditing and that is one of the 338 00:16:36,280 --> 00:16:37,840 things that I think would help with. 339 00:16:37,840 --> 00:16:40,600 I don't think I know I've experienced that helping with 340 00:16:41,080 --> 00:16:43,680 delivering audit reports that actually provide value because 341 00:16:43,680 --> 00:16:46,400 the scope of the audit is focused on what's going to add 342 00:16:46,400 --> 00:16:48,240 value to the organization and the clients. 343 00:16:49,060 --> 00:16:52,020 I think that's the perfect segue to go into your concept, right? 344 00:16:52,060 --> 00:16:54,780 So explain to us a little bit more about this auditing with 345 00:16:54,780 --> 00:16:57,420 agility. Is this just some application of 346 00:16:57,500 --> 00:17:01,140 agile methodology to some other parts of non technology? 347 00:17:01,140 --> 00:17:02,980 So tell us more about it. Yeah. 348 00:17:02,980 --> 00:17:04,460 So you're on the right path there. 349 00:17:04,780 --> 00:17:07,140 The traditional way of auditing is a waterfall approach. 350 00:17:07,140 --> 00:17:09,619 So that stage gated approach that is similar to software 351 00:17:09,619 --> 00:17:11,579 development, waterfall and software development. 352 00:17:11,780 --> 00:17:14,780 You do one stage before you go to the next stage, before you go 353 00:17:14,780 --> 00:17:17,500 to the next stage and you're very heads down in each of those 354 00:17:17,500 --> 00:17:19,990 stages. So we were finding a lot of 355 00:17:19,990 --> 00:17:21,710 those challenges that we talked about. 356 00:17:21,790 --> 00:17:25,910 We, the auditing profession, not just Clarissa and her daily 357 00:17:25,910 --> 00:17:29,550 struggles, but the auditing profession realized that things 358 00:17:29,550 --> 00:17:31,790 were changing. This waterfall approach, strict 359 00:17:31,790 --> 00:17:34,710 framework that we have to do this very sequential thing and 360 00:17:34,710 --> 00:17:39,030 every situation wasn't keeping up with the environment that all 361 00:17:39,030 --> 00:17:40,590 of our organizations were working in. 362 00:17:40,750 --> 00:17:44,590 So we also saw that in the technology world and business 363 00:17:44,590 --> 00:17:48,290 world, people were applying. Agile concepts and seeing 364 00:17:48,290 --> 00:17:51,170 success. So we moved to There was a big 365 00:17:51,170 --> 00:17:53,290 movement for what's called Agile auditing. 366 00:17:53,650 --> 00:17:57,370 Agile auditing is pretty much applying A Scrum framework to 367 00:17:57,370 --> 00:18:00,210 the audit process. So you've got sprints, typically 368 00:18:00,210 --> 00:18:03,250 about two weeks, you've got Scrum masters, daily standups. 369 00:18:03,490 --> 00:18:05,690 All of the things that you'll see in a Scrum framework applied 370 00:18:05,690 --> 00:18:08,850 to internal auditing. And just like with waterfall, it 371 00:18:08,850 --> 00:18:10,970 was do the same thing all the time. 372 00:18:10,970 --> 00:18:13,130 So do sprints all the time, do your daily standups all the 373 00:18:13,130 --> 00:18:16,220 time, in every situation. And some organizations found a 374 00:18:16,220 --> 00:18:19,060 lot of success with that. My own personal experience, I 375 00:18:19,060 --> 00:18:21,780 found a lot of success with that in certain parts of the 376 00:18:21,780 --> 00:18:24,500 organization. So auditing technology. 377 00:18:24,780 --> 00:18:28,300 Some of my clients leveraged Scrum frameworks to manage their 378 00:18:28,300 --> 00:18:30,060 own work. So we were able to fit right in 379 00:18:30,060 --> 00:18:33,620 there and deliver our audits and sprints in those situations and 380 00:18:33,620 --> 00:18:36,340 it was amazing. But there were also situations 381 00:18:36,340 --> 00:18:39,700 where that didn't work out quite as well, so. 382 00:18:40,020 --> 00:18:42,180 I started thinking, you know, we started thinking, okay do we 383 00:18:42,180 --> 00:18:44,380 want to do agile auditing or not? 384 00:18:44,380 --> 00:18:47,500 And it was very binary, like you have to pick waterfall or you 385 00:18:47,500 --> 00:18:50,540 have to pick agile auditing. And we were doing agile. 386 00:18:51,060 --> 00:18:55,220 And it kind of dawned on me that the whole point of as I was 387 00:18:55,420 --> 00:18:58,620 attending more conferences related to IT and working 388 00:18:58,620 --> 00:19:01,300 DevOps, ways of working and agile ways of working, reading 389 00:19:01,300 --> 00:19:05,100 about business agility, I was really realizing that we were 390 00:19:05,100 --> 00:19:08,860 falling into a trap of doing agile instead of being agile. 391 00:19:09,270 --> 00:19:11,390 We were looking for a framework because we're auditors. 392 00:19:11,390 --> 00:19:13,190 We like frameworks. We started out with checklists 393 00:19:13,190 --> 00:19:15,950 like it's comfortable. But again, that's not working 394 00:19:15,950 --> 00:19:17,470 today. I mean it's working. 395 00:19:17,510 --> 00:19:19,670 It's got so many opportunities to be so much better. 396 00:19:19,670 --> 00:19:21,390 Like I don't want to be the bad guy anymore. 397 00:19:21,390 --> 00:19:23,270 I don't want you to run from me as an auditor. 398 00:19:23,270 --> 00:19:25,830 I want you to call me up and say like, hey, I've got a question. 399 00:19:26,070 --> 00:19:27,910 I need audit perspective. Can you help me? 400 00:19:28,270 --> 00:19:31,390 So following in the trap of doing agile versus being agile, 401 00:19:31,390 --> 00:19:34,390 so started experimenting with what I call auditing with 402 00:19:34,390 --> 00:19:38,230 agility and it sounds very similar to agile auditing. 403 00:19:38,670 --> 00:19:41,430 But instead of agile auditing, when people hear that, they 404 00:19:41,430 --> 00:19:44,070 think it's a thing to do. When you hear auditing with 405 00:19:44,070 --> 00:19:46,590 Agility, I think it's more clear that you're auditing. 406 00:19:46,710 --> 00:19:48,870 That's what you do. You're not changing what you do, 407 00:19:49,230 --> 00:19:50,950 but you're doing it with agility. 408 00:19:51,270 --> 00:19:54,750 It's a very minor tweak in words, but it's very 409 00:19:54,750 --> 00:19:56,670 intentional. It's trying to get the point out 410 00:19:56,670 --> 00:19:58,790 that it's not something you do. It's not this framework that 411 00:19:58,790 --> 00:20:01,270 you're going to cookie cutter apply in every situation. 412 00:20:01,470 --> 00:20:03,270 We're still auditing. We're still providing that 413 00:20:03,270 --> 00:20:05,270 assurance that things are working right or that you're 414 00:20:05,270 --> 00:20:07,310 going to identify things when they don't work right. 415 00:20:07,750 --> 00:20:10,710 We're just doing that in a more flexible approach that anchors 416 00:20:10,710 --> 00:20:14,710 back to those agile principles instead of specific frameworks. 417 00:20:14,990 --> 00:20:17,870 And then it also incorporates, because I was heavily influenced 418 00:20:17,870 --> 00:20:20,830 by these DevOps Enterprise summits and the talented 419 00:20:20,830 --> 00:20:24,390 speakers there explaining super highly technical things that 420 00:20:24,390 --> 00:20:26,110 most of the time we're way over my head. 421 00:20:26,110 --> 00:20:29,790 But I was picking up a lot of their ways of working and the 422 00:20:29,790 --> 00:20:33,430 success they were seeing through applying that DevOps mindset. 423 00:20:33,880 --> 00:20:37,800 Also, what resonated with me and kind of why auditing it with 424 00:20:37,800 --> 00:20:40,520 agility, I think is, you know, really where organizations need 425 00:20:40,520 --> 00:20:44,200 to go is after I did one of my presentations. 426 00:20:44,760 --> 00:20:47,360 It wasn't in 2020, I think it was in 2021. 427 00:20:47,600 --> 00:20:51,600 I started talking about applying some of these DevOps concepts to 428 00:20:51,760 --> 00:20:53,800 internal auditing. It was kind of the birth of 429 00:20:53,800 --> 00:20:57,480 auditing with agility and Gene Kim when I submitted my 430 00:20:57,600 --> 00:21:00,870 presentation for that, he said. He was really impressed by it 431 00:21:00,870 --> 00:21:06,070 and he said this is very similar to the 2009 presentation that 432 00:21:06,070 --> 00:21:09,950 John Alspa and Paul Hammond did about Flickr and that was kind 433 00:21:09,950 --> 00:21:12,110 of the birth of DevOps. So this is kind of awesome 434 00:21:12,110 --> 00:21:14,830 because it was the birth of auditing with agility and. 435 00:21:15,310 --> 00:21:18,150 We had not seen the 2009 presentation at that point. 436 00:21:18,310 --> 00:21:21,950 So I went and I watched it and it was so cool to see. 437 00:21:21,950 --> 00:21:25,230 It was operations team and the developers, they're not getting 438 00:21:25,230 --> 00:21:27,110 along and they're not incentivized to do the same 439 00:21:27,110 --> 00:21:28,750 thing. They're incentivized kind of to 440 00:21:28,750 --> 00:21:32,390 get in each other's way, very similar to audits and clients. 441 00:21:32,390 --> 00:21:34,670 You know, clients are trying to do their thing and here come the 442 00:21:34,670 --> 00:21:36,750 auditors get in their way and we're just trying to get an 443 00:21:36,750 --> 00:21:38,510 audit report out. But management's doing these 444 00:21:38,510 --> 00:21:40,190 things and not sending us the right things that we're 445 00:21:40,190 --> 00:21:42,950 requesting. So it was really, really cool to 446 00:21:42,950 --> 00:21:46,190 see that. Those parallels and then how 447 00:21:46,190 --> 00:21:50,110 DevOps got the two of those groups who historically didn't 448 00:21:50,110 --> 00:21:51,910 work so well together to work together. 449 00:21:52,190 --> 00:21:54,150 That's what I'm trying to do with auditing with Agility is 450 00:21:54,150 --> 00:21:55,870 trying to get auditors and clients to get out of each 451 00:21:55,870 --> 00:21:58,630 other's way and work together and help each other. 452 00:21:59,310 --> 00:22:01,830 That was a long, long explanation, right? 453 00:22:02,750 --> 00:22:05,710 I think that's really exciting, especially again like coming 454 00:22:05,710 --> 00:22:08,390 back to you mentioned about DevOps Enterprise Summit, right. 455 00:22:08,390 --> 00:22:10,590 I think that also picked my interest when I read your book, 456 00:22:10,590 --> 00:22:14,390 the parallels between your presentation and the 209 John 457 00:22:14,390 --> 00:22:17,870 Osborne presentation, the first moment where we all get 458 00:22:17,870 --> 00:22:20,870 introduced into DevOps, you know, so many deploys per days 459 00:22:20,870 --> 00:22:23,480 and things like that. So I really love the parallels 460 00:22:23,480 --> 00:22:27,160 that you bring here, which brings us to the concept of why 461 00:22:27,160 --> 00:22:29,840 DevOps is needed. So the first traditionally in 462 00:22:29,840 --> 00:22:33,600 the 1st place, right, people try to create a silo between 463 00:22:33,600 --> 00:22:37,080 development and operations and the functions actually kind of 464 00:22:37,080 --> 00:22:39,880 like different if you look from the traditional perspective. 465 00:22:39,880 --> 00:22:43,160 One is to introduce more change, the other is to actually control 466 00:22:43,160 --> 00:22:45,120 change. I believe this is the same thing 467 00:22:45,120 --> 00:22:47,840 that happens in the audit and the clients, let's call it 468 00:22:47,840 --> 00:22:50,520 client as well. So client always wants to do 469 00:22:50,520 --> 00:22:52,520 their own business, you know, introduce change, create new 470 00:22:52,520 --> 00:22:55,560 products, create new systems, whatever that is while audited, 471 00:22:55,560 --> 00:22:58,840 try to manage the risk, the control and things like that. 472 00:22:59,120 --> 00:23:02,800 So when you took this parallels right, what would be some of the 473 00:23:02,800 --> 00:23:06,720 interesting things that Jin see in your presentation that 474 00:23:06,840 --> 00:23:09,960 probably will become a birth of something new in the future? 475 00:23:10,970 --> 00:23:14,890 Yeah, a lot of it was. So in my presentation there, I 476 00:23:14,890 --> 00:23:17,970 was representing audit and I was copresenting. 477 00:23:17,970 --> 00:23:21,290 So historically, up to that point, I had copresented with 478 00:23:21,490 --> 00:23:24,370 other auditors and this was the first time that I was 479 00:23:24,370 --> 00:23:26,090 copresenting with one of my clients. 480 00:23:26,410 --> 00:23:29,730 And similar to the 2009 presentation, it was somebody 481 00:23:29,730 --> 00:23:32,610 from development and somebody from operations sharing the 482 00:23:32,610 --> 00:23:35,370 stage. And my client and I had a lot of 483 00:23:35,370 --> 00:23:37,730 fun too. I mean, I think work should be 484 00:23:37,730 --> 00:23:40,330 fun. I love having fun when I work. 485 00:23:40,810 --> 00:23:44,570 So us having the presentation and you could tell we had a 486 00:23:44,570 --> 00:23:47,730 great relationship. We had a lot of fun doing the 487 00:23:47,730 --> 00:23:50,330 presentation that really paralleled with that. 488 00:23:50,730 --> 00:23:53,690 And then with the 2009 presentation, the two 489 00:23:53,690 --> 00:23:56,330 presenters, one from development, one from operations 490 00:23:56,330 --> 00:24:00,930 talked about how at their organization they were able to 491 00:24:01,090 --> 00:24:03,850 break down those silos, break down those barriers, have a 492 00:24:03,850 --> 00:24:05,930 common objective and work together. 493 00:24:06,370 --> 00:24:10,090 Very similar to what my client and I were explaining in our 494 00:24:10,090 --> 00:24:14,370 presentation to so auditor, audit client, typically butting 495 00:24:14,370 --> 00:24:17,650 heads, not getting along super well or just tolerating each 496 00:24:17,650 --> 00:24:21,650 other to get through an audit. And we talked about how we 497 00:24:21,650 --> 00:24:24,330 worked as one team. So it wasn't the auditor team 498 00:24:24,330 --> 00:24:27,450 and the client team, it was one team, the team and we were very 499 00:24:27,450 --> 00:24:30,410 like specific when we would say very intentional when we say 500 00:24:30,410 --> 00:24:34,730 like the team, all of us not you know you over there in us here 501 00:24:34,730 --> 00:24:37,080 we were one team. We did have our separate 502 00:24:37,080 --> 00:24:39,800 reporting structures as is needed for us as auditors to 503 00:24:39,800 --> 00:24:42,360 maintain our independence, but that doesn't mean we cannot work 504 00:24:42,640 --> 00:24:47,280 together closely as one team. We worked so closely together to 505 00:24:47,280 --> 00:24:51,400 make sure that we were aligned on what our common objective was 506 00:24:51,520 --> 00:24:55,560 and it was provide and get insights about the most 507 00:24:55,560 --> 00:24:57,680 important things in that particular area. 508 00:24:58,000 --> 00:25:03,000 So really those two primary differentiators were the huge 509 00:25:03,000 --> 00:25:06,060 parallels between the two. You mentioned something about 510 00:25:06,060 --> 00:25:08,780 different reporting line, right. So I think in the world we 511 00:25:08,780 --> 00:25:11,740 always have this thing called segregation of duty maker and 512 00:25:11,740 --> 00:25:13,700 checker. I think that is also what 513 00:25:13,700 --> 00:25:17,820 happened before the DevOps world and someone needs to have like a 514 00:25:17,820 --> 00:25:21,620 different maybe a like access control or approval yeah, before 515 00:25:21,620 --> 00:25:23,260 some change can go into production. 516 00:25:23,500 --> 00:25:25,140 I think similar thing in audit as well. 517 00:25:25,140 --> 00:25:28,180 So how do you see this segregation of duty now with 518 00:25:28,180 --> 00:25:30,540 your auditing with agility concept? 519 00:25:31,250 --> 00:25:33,970 Yeah, this is a common question and this was one that really 520 00:25:33,970 --> 00:25:38,170 sparked me getting into these DevOps enterprise summits and 521 00:25:38,170 --> 00:25:40,850 presentations. So the question that we would 522 00:25:40,850 --> 00:25:43,970 get, that I would get from my clients is how do I pass an 523 00:25:43,970 --> 00:25:49,010 audit when we're using DevOps and we're not doing segregation 524 00:25:49,010 --> 00:25:51,570 of duties through the access controls or how we historically 525 00:25:51,570 --> 00:25:53,770 would. And I mean this was my kind of 526 00:25:53,770 --> 00:25:56,130 my first view into the misconceptions like there's no 527 00:25:56,130 --> 00:25:58,370 passing and audit. I don't have a pass fail. 528 00:25:58,370 --> 00:26:01,300 I don't have like a. Big green check mark to provide 529 00:26:01,300 --> 00:26:03,900 at the end of an audit, but then also thinking through 530 00:26:03,900 --> 00:26:06,900 segregations of duties and being new to the role. 531 00:26:07,260 --> 00:26:10,980 I and I still ask like very elementary questions which has 532 00:26:10,980 --> 00:26:13,220 turned out to be a strength of mine and something that has 533 00:26:13,260 --> 00:26:15,940 added value. But here I am a couple days into 534 00:26:15,940 --> 00:26:19,300 my new role leading technology audit and I was like why do we 535 00:26:19,300 --> 00:26:21,610 segregate duties? And the first answer was because 536 00:26:21,610 --> 00:26:23,570 the auditors told us we had to. I'm like, ooh, try again. 537 00:26:23,570 --> 00:26:25,650 Don't do things just because auditors tell you if you don't 538 00:26:25,650 --> 00:26:28,010 understand why we're telling you to do something, challenge us 539 00:26:28,010 --> 00:26:30,730 because you should never do something just because the 540 00:26:30,730 --> 00:26:33,410 auditors want you to do it. The auditors should be able to 541 00:26:33,410 --> 00:26:37,050 explain, we want you to do this because here's the risk and you 542 00:26:37,050 --> 00:26:40,010 need to control that risk in accordance with your risk 543 00:26:40,090 --> 00:26:42,610 appetite and tolerance. So when we really started 544 00:26:42,610 --> 00:26:45,290 peeling back the layers of why do we segregate duties? 545 00:26:45,860 --> 00:26:48,700 We started thinking about things like we want to make sure that 546 00:26:48,700 --> 00:26:50,980 somebody doesn't introduce something into production that's 547 00:26:50,980 --> 00:26:54,220 going to do bad things. Megan is like super not 548 00:26:54,220 --> 00:26:55,820 technical. So bear with me there. 549 00:26:56,300 --> 00:27:00,900 So I'm like, OK, so historically we have managed that risk by not 550 00:27:00,900 --> 00:27:04,140 letting the same person push their stuff through without 551 00:27:04,140 --> 00:27:08,260 having somebody else give the OK, we've segregated duties. 552 00:27:08,700 --> 00:27:12,100 So then I challenged the group and was asking, OK, what else 553 00:27:12,100 --> 00:27:15,270 could we do? Manage that risk without having 554 00:27:15,430 --> 00:27:18,070 two separate access lists. And that's when we really 555 00:27:18,070 --> 00:27:20,390 started understanding. OK, well, maybe we could have 556 00:27:20,390 --> 00:27:23,910 automated checks and I can push my stuff through and it'll go 557 00:27:23,910 --> 00:27:27,510 through only when this automated test says it passes all these 558 00:27:27,510 --> 00:27:29,430 things the same thing that a human would do when they're 559 00:27:29,430 --> 00:27:31,780 looking at the code. Or the change or whatever it is, 560 00:27:31,780 --> 00:27:34,700 if this automated test, it passes that test, then it goes 561 00:27:34,700 --> 00:27:37,020 through and essentially you've segregated the duties not 562 00:27:37,020 --> 00:27:40,620 between two people, but between the person wanting to promote 563 00:27:40,620 --> 00:27:43,100 the code, the developer, and an automated test. 564 00:27:43,420 --> 00:27:46,220 So that was one example. There's other examples, but it 565 00:27:46,220 --> 00:27:49,340 was really about thinking through, getting rid of that 566 00:27:49,340 --> 00:27:52,540 checklist of we need to look for segregation of duties, working 567 00:27:52,540 --> 00:27:54,980 with our clients to understand what are you trying to 568 00:27:54,980 --> 00:27:56,740 accomplish. We're trying to make sure things 569 00:27:56,740 --> 00:27:59,980 get into production so that we can help serve our business. 570 00:28:00,420 --> 00:28:02,500 What could go wrong? What are the risks? 571 00:28:02,700 --> 00:28:05,860 We could get something in there that does bad things, either 572 00:28:05,860 --> 00:28:07,260 intentionally or unintentionally. 573 00:28:07,260 --> 00:28:10,580 People make mistakes, Okay. What can we do? 574 00:28:10,580 --> 00:28:12,500 Or what ways can you manage that risk? 575 00:28:12,500 --> 00:28:14,140 And what ways do you manage that risk? 576 00:28:14,620 --> 00:28:16,620 So instead of walking in and saying, I need to see 577 00:28:16,620 --> 00:28:19,220 segregation of duties, give me your access lists and you give 578 00:28:19,220 --> 00:28:21,180 me your access lists. And I tell you, well, these 579 00:28:21,260 --> 00:28:23,340 people have access to do both. You're like, why? 580 00:28:23,340 --> 00:28:25,700 No, it's set up that way. Like, that's a waste of time. 581 00:28:26,100 --> 00:28:28,340 Instead, we're understanding what you're trying to 582 00:28:28,340 --> 00:28:31,260 accomplish, what can go wrong, how you're controlling that 583 00:28:31,260 --> 00:28:32,620 risk, how you're managing that risk. 584 00:28:32,620 --> 00:28:35,260 And then we test that. So then instead of looking at 585 00:28:35,260 --> 00:28:37,980 the access list, we're going to look at how is this test set up, 586 00:28:37,980 --> 00:28:40,340 this automated test set up, How's it designed? 587 00:28:40,380 --> 00:28:43,740 Is it designed the same way to look for the same things that a 588 00:28:43,740 --> 00:28:46,380 peer reviewer would? Or, you know, in a world where 589 00:28:46,380 --> 00:28:49,180 those duties are segregated? And then is it operating the way 590 00:28:49,180 --> 00:28:51,620 that you think it is? So it's supposed to identify 591 00:28:51,620 --> 00:28:54,780 these things and not let it go through to production if it 592 00:28:54,780 --> 00:28:56,530 doesn't? Meet these criteria. 593 00:28:56,530 --> 00:28:59,090 Is it doing that? Is it letting things through 594 00:28:59,170 --> 00:29:02,290 when it's supposed to? So if it passes all of these 595 00:29:02,290 --> 00:29:04,490 tests, it's supposed to go to production. 596 00:29:04,610 --> 00:29:06,730 We would test that. And that's going to provide a 597 00:29:06,730 --> 00:29:09,610 lot more value than to your earlier point, us handing you a 598 00:29:09,610 --> 00:29:12,250 report that says you don't have segregation of duties in place. 599 00:29:12,250 --> 00:29:13,810 What are you supposed to do with that? 600 00:29:14,050 --> 00:29:15,290 That's something you're hanging on your fridge. 601 00:29:17,840 --> 00:29:19,640 Right. I really left a new set in the 602 00:29:19,640 --> 00:29:23,200 beginning that we just follow whatever auditor set sometimes 603 00:29:23,280 --> 00:29:26,320 that was what happened. I think in most of the client 604 00:29:26,320 --> 00:29:29,800 situations, we just follow whatever auditors say because 605 00:29:29,800 --> 00:29:32,640 maybe they come from a compliance point of view or they 606 00:29:32,640 --> 00:29:35,280 come from a standardized practices and things like that. 607 00:29:35,280 --> 00:29:38,000 But always ask or maybe challenge right, why we need to 608 00:29:38,000 --> 00:29:40,540 do certain things. Because sometimes the context is 609 00:29:40,540 --> 00:29:43,380 different and like you said, probably we could do a better 610 00:29:43,380 --> 00:29:46,420 way instead of just following word by word what the auditor 611 00:29:46,420 --> 00:29:48,420 said. Or maybe you are doing it in a 612 00:29:48,420 --> 00:29:50,500 different way. So maybe the finding is you 613 00:29:50,500 --> 00:29:52,860 don't have duties segregated, but you do have these automated 614 00:29:52,860 --> 00:29:56,300 tests in place. So instead of having a finding 615 00:29:56,420 --> 00:29:58,620 or an audit report that says you have to segregate duties and 616 00:29:58,620 --> 00:30:02,740 just now segregating duties, you can educate your auditors on 617 00:30:02,860 --> 00:30:04,220 this is how we're managing that risk. 618 00:30:04,220 --> 00:30:07,100 Let me walk you through this. So yeah, I just wanted to point 619 00:30:07,100 --> 00:30:10,090 that out too. Yeah, I think it all comes back 620 00:30:10,090 --> 00:30:12,170 to the controls that you want in place, right? 621 00:30:12,170 --> 00:30:15,690 So not necessarily the technique or the tactics, right, whatever 622 00:30:15,690 --> 00:30:17,930 that is. How you're managing that risk 623 00:30:18,250 --> 00:30:20,730 and bringing your auditors along so that they understand it? 624 00:30:21,170 --> 00:30:24,410 Right, so let's go to your in depth about your concept 625 00:30:24,410 --> 00:30:26,210 auditing with agility. You mentioned there are three 626 00:30:26,210 --> 00:30:29,090 values, so the first one is value driven Auditing. 627 00:30:29,290 --> 00:30:31,730 The second one is Integrated Auditing 2.0. 628 00:30:31,930 --> 00:30:33,490 It's interesting there's a 2.0 there. 629 00:30:33,770 --> 00:30:36,930 And adaptable auditing. So maybe we can just go through 630 00:30:36,930 --> 00:30:39,890 scheme, some of them one by one Value driven auditing. 631 00:30:39,890 --> 00:30:42,050 What do you mean by this? Yep. 632 00:30:42,130 --> 00:30:44,130 So this is it. Gets back to that point and 633 00:30:44,130 --> 00:30:47,260 solves that problem of. You getting a report that's an 634 00:30:47,260 --> 00:30:48,980 audit report, that's not valuable to you. 635 00:30:49,420 --> 00:30:52,980 So value driven auditing is really going to make sure that 636 00:30:52,980 --> 00:30:55,020 the audit scope. So what the auditors are going 637 00:30:55,020 --> 00:30:57,220 to look at and what they're going to do is going to add 638 00:30:57,340 --> 00:30:59,580 value to the organization. So it's going to be anchored 639 00:30:59,580 --> 00:31:03,420 back to what's most important to the organization and its key 640 00:31:03,420 --> 00:31:05,740 stakeholders which include the audit clients. 641 00:31:06,080 --> 00:31:08,800 So we're going to look at where are the biggest risks or where 642 00:31:08,800 --> 00:31:10,200 are the greatest opportunities, too. 643 00:31:10,200 --> 00:31:14,360 So there's risk in not doing things and there's risk in doing 644 00:31:14,360 --> 00:31:16,280 things. So value driven auditing is 645 00:31:16,280 --> 00:31:19,640 really just anchoring back to what is going to add the most 646 00:31:19,640 --> 00:31:23,160 value to the organization and focusing the work there. 647 00:31:23,560 --> 00:31:26,800 And we do talk through a number of practices that you can 648 00:31:26,800 --> 00:31:29,920 implement to achieve that value driven auditing. 649 00:31:30,240 --> 00:31:32,520 But I first just want to focus on like, what are those 3 core 650 00:31:32,520 --> 00:31:34,200 components? Let's define those and then we 651 00:31:34,200 --> 00:31:36,560 can. Dive into some that I think the 652 00:31:36,560 --> 00:31:39,440 audience today are really going to benefit from. 653 00:31:39,640 --> 00:31:41,680 So Yep, value driven auditing first. 654 00:31:42,000 --> 00:31:44,280 I know you mentioned integrating auditing 2.0 and we're 655 00:31:44,280 --> 00:31:46,000 interested in the 2.0 piece of that. 656 00:31:46,480 --> 00:31:49,800 So in the auditing world, it's probably been more than a few 657 00:31:49,800 --> 00:31:52,640 years ago, but audits used to be performed. 658 00:31:52,640 --> 00:31:55,080 You'd have a compliance audit, you'd have an operational audit 659 00:31:55,080 --> 00:31:57,440 and you'd have an IT audit and then those would all be 660 00:31:57,440 --> 00:32:00,320 delivered separately or they'd stitch them together at the end 661 00:32:00,600 --> 00:32:03,280 in one report, but all the work would be performed separately. 662 00:32:03,860 --> 00:32:06,740 So the auditing profession started doing what's called 663 00:32:06,740 --> 00:32:08,940 integrated auditing and you would have all of those auditors 664 00:32:08,940 --> 00:32:11,660 on the same audit. So each audit would have a 665 00:32:11,660 --> 00:32:15,780 compliance operation on IT lens which really helped and breaking 666 00:32:15,780 --> 00:32:18,260 down those silos within the audit function and provide a 667 00:32:18,380 --> 00:32:23,900 more holistic view and better view of the environment than the 668 00:32:23,940 --> 00:32:25,820 separate audits being stitched together. 669 00:32:26,140 --> 00:32:27,820 That's not what I go into in the book. 670 00:32:28,060 --> 00:32:29,700 That is should be a given right now. 671 00:32:29,700 --> 00:32:32,260 We should all be there. So what I mean by integrated 672 00:32:32,260 --> 00:32:35,300 auditing 2.0 is it's kind of taking that to the next level. 673 00:32:35,820 --> 00:32:39,380 And what we do here is we're integrating audit work with our 674 00:32:39,380 --> 00:32:42,140 audit clients work and we still maintain that independence. 675 00:32:42,140 --> 00:32:44,500 I know that's a question I get a lot from auditors is like how 676 00:32:44,500 --> 00:32:46,380 can we do this and still be independent. 677 00:32:46,620 --> 00:32:48,620 There's plenty of ways that we can do that and still be 678 00:32:48,620 --> 00:32:51,060 independent. Even the institute of internal 679 00:32:51,060 --> 00:32:54,340 auditors who is our governing body, they set the standards for 680 00:32:54,340 --> 00:32:57,230 internal auditing. They tell us that independence 681 00:32:57,230 --> 00:33:00,150 doesn't mean isolation, so you don't have to have working 682 00:33:00,150 --> 00:33:01,950 silos. We still have that different 683 00:33:01,950 --> 00:33:04,590 reporting structure, we still maintain those decision rights. 684 00:33:04,870 --> 00:33:08,350 But what we focus on with this 2.0 version of integrated 685 00:33:08,350 --> 00:33:11,110 auditing is integrating audit work with clients work. 686 00:33:11,730 --> 00:33:16,290 The third component is adaptable auditing and this is where we 687 00:33:16,290 --> 00:33:18,330 build in the ability to respond to change. 688 00:33:18,410 --> 00:33:21,050 So we're going to have a flexible process to audit 689 00:33:21,050 --> 00:33:22,530 instead of this strict framework. 690 00:33:22,570 --> 00:33:25,050 We're going to be able to pivot. We're going to be able to 691 00:33:25,050 --> 00:33:27,050 understand when we should stop auditing. 692 00:33:27,330 --> 00:33:30,410 So with our old audit waterfall approach, we would have our plan 693 00:33:30,410 --> 00:33:32,930 and we would go heads down and execute it and not come up for 694 00:33:32,930 --> 00:33:35,520 air until the end. And we really miss opportunities 695 00:33:35,520 --> 00:33:38,560 to determine do we still need to go down this path or do we know 696 00:33:38,560 --> 00:33:41,400 enough to deliver now and get out of our clients hair and move 697 00:33:41,400 --> 00:33:44,720 on to something else. So adaptable auditing is where 698 00:33:44,720 --> 00:33:47,640 we have that flexibility, the ability to respond to change 699 00:33:47,640 --> 00:33:51,160 which is super important in today's crazy fast changing 700 00:33:51,160 --> 00:33:54,480 environment. Thanks for a quick overview of 701 00:33:54,480 --> 00:33:57,120 the three values of the auditing with agility. 702 00:33:57,280 --> 00:33:59,920 So like you said right, the 1st that picked my interest is 703 00:33:59,920 --> 00:34:01,560 actually the integrated auditing. 704 00:34:01,900 --> 00:34:03,740 Regardless 2.0 or not, right? Because I don't. 705 00:34:03,820 --> 00:34:05,900 I don't know the. History of the auditing, so 706 00:34:05,900 --> 00:34:08,460 specifically you mentioned that integrating audit work with the 707 00:34:08,460 --> 00:34:11,219 client's work. So does it mean that auditors 708 00:34:11,219 --> 00:34:14,179 now have a place in the team like you have a dedicated 709 00:34:14,179 --> 00:34:17,739 auditors as part of the team that instead of thinking about 710 00:34:17,739 --> 00:34:20,500 business stories, right, we call stories in the tech world 711 00:34:20,739 --> 00:34:22,820 business stories or business requirements, but you also have 712 00:34:22,820 --> 00:34:25,500 a like an audit kind of a stories, audit requirements as 713 00:34:25,500 --> 00:34:27,699 part of the work. Maybe tell us a little bit more 714 00:34:27,699 --> 00:34:30,340 on that. There's a bunch of practices 715 00:34:30,340 --> 00:34:34,219 that you can implement and here is where I think the audit 716 00:34:34,219 --> 00:34:38,580 clients have a huge opportunity to influence a better audit 717 00:34:38,580 --> 00:34:41,300 experience. So a lot of people think, OK, 718 00:34:41,300 --> 00:34:44,300 beyond edge auditing, this is primarily for auditors to read. 719 00:34:44,580 --> 00:34:47,739 It's got two primary audit audiences, both auditors and 720 00:34:47,739 --> 00:34:50,420 clients, because just like DevOps, the developers couldn't 721 00:34:50,420 --> 00:34:53,060 do DevOps by themselves. Neither could operations. 722 00:34:53,060 --> 00:34:56,420 They both needed to go and implement those concepts. 723 00:34:56,420 --> 00:34:58,940 It's the same as here and integrated auditing, I think 724 00:34:58,940 --> 00:35:02,340 it's a great place for clients to start and start influencing 725 00:35:02,340 --> 00:35:04,860 that experience. So you mentioned in the 726 00:35:04,860 --> 00:35:07,540 question, do you have a dedicated auditor, You could, 727 00:35:07,820 --> 00:35:10,620 but you also don't know, you know that the type of work, you 728 00:35:10,620 --> 00:35:12,740 don't know when it's going to be there and when it's not. 729 00:35:13,020 --> 00:35:17,300 So there's some things that we can do is like my clients know 730 00:35:17,300 --> 00:35:19,740 that they can call me anytime with questions and they do. 731 00:35:20,130 --> 00:35:22,490 I'll get a random hang on a Tuesday afternoon. 732 00:35:22,490 --> 00:35:25,050 Hey, do you have a minute for a quick call and they'll call me 733 00:35:25,050 --> 00:35:26,250 up? Hey, I'm going through this. 734 00:35:26,570 --> 00:35:28,650 I wanted you to get your thoughts on should I think about 735 00:35:28,650 --> 00:35:31,250 it this way or that way or, you know, they're looking for advice 736 00:35:31,450 --> 00:35:34,490 and I can give that advice and then they go on their way and I 737 00:35:34,490 --> 00:35:36,410 go on my way. So it's not a full blown audit 738 00:35:36,810 --> 00:35:41,210 being able to just call somebody up and get that real time 739 00:35:41,210 --> 00:35:43,860 feedback from them is. Super helpful. 740 00:35:43,860 --> 00:35:46,540 And that's part of the integrated auditing feedback 741 00:35:46,540 --> 00:35:48,980 loops. So regular feedback loops, real 742 00:35:48,980 --> 00:35:53,100 time feedback loops. Those are probably the most 743 00:35:53,100 --> 00:35:56,740 straightforward thing that I can think of for clients to start 744 00:35:57,100 --> 00:36:00,060 implementing. That is, you know you don't have 745 00:36:00,060 --> 00:36:01,500 to wait for the auditors to reach out to you. 746 00:36:01,540 --> 00:36:04,340 You can implement and you can start a feedback loop so you 747 00:36:04,340 --> 00:36:05,580 have a question for your auditor. 748 00:36:05,580 --> 00:36:07,980 Call them up and it may be intimidating, especially at 749 00:36:07,980 --> 00:36:09,740 first. If you don't have that working 750 00:36:09,740 --> 00:36:13,100 relationship with them yet and you're afraid this is going to 751 00:36:13,100 --> 00:36:15,780 trigger a huge audit and it's going to be a bunch of extra 752 00:36:15,780 --> 00:36:19,500 time spent, you could start by figuring out what do you need 753 00:36:19,500 --> 00:36:22,460 from an audit, what do you need from your auditors and set up 754 00:36:22,460 --> 00:36:24,700 coffee with them. Virtual coffee, Real coffee. 755 00:36:24,700 --> 00:36:27,220 I love coffee, so you know, it's one of my favorite things. 756 00:36:27,740 --> 00:36:30,340 But just start that feedback loop of, hey, we had this audit 757 00:36:30,340 --> 00:36:31,860 or I know we've got this audit coming up. 758 00:36:32,060 --> 00:36:33,980 I'd love to see us do this in it. 759 00:36:34,300 --> 00:36:38,340 I'd love to see a focus on this particular area or you know 760 00:36:38,340 --> 00:36:40,100 what's really keeping me up at night this? 761 00:36:40,420 --> 00:36:42,500 Can we spend some time talking about that? 762 00:36:42,900 --> 00:36:45,180 Or even if you don't have an audit coming up, just here are 763 00:36:45,180 --> 00:36:48,260 some things that I've got questions on or I'd love to see 764 00:36:48,260 --> 00:36:51,100 from my auditors. Feedback can even be that 765 00:36:51,100 --> 00:36:54,060 clients reach out to me and say I'd love you to attend our OPS 766 00:36:54,060 --> 00:36:57,540 review meeting so that you can help us stay on top of the open 767 00:36:57,540 --> 00:37:00,660 findings we have because we sometimes lose sight of those. 768 00:37:00,660 --> 00:37:03,540 Sure, absolutely. Not only am I connecting with 769 00:37:03,540 --> 00:37:05,980 them and providing them information on open audit 770 00:37:05,980 --> 00:37:08,180 findings, but I'm also learning more about what's important to 771 00:37:08,180 --> 00:37:10,620 them. So I have this idea of what I 772 00:37:10,620 --> 00:37:13,420 want to audit in that space, but they're spending all this time 773 00:37:13,420 --> 00:37:15,140 and all this money on this one thing. 774 00:37:15,690 --> 00:37:18,490 Hey, could you use some objective advice as you're 775 00:37:18,530 --> 00:37:21,210 building that out? Yeah, that would be great. 776 00:37:21,210 --> 00:37:24,170 So feedback loops are super, super helpful. 777 00:37:24,610 --> 00:37:27,210 Another thing about feedback loops is if you don't provide 778 00:37:27,210 --> 00:37:29,930 feedback to your auditors on what a better audit experience 779 00:37:29,930 --> 00:37:32,330 looks like, they're not going to know to make a change. 780 00:37:32,690 --> 00:37:34,730 Or they might know that they should make a change, but they 781 00:37:34,730 --> 00:37:37,130 might try a bunch of things that aren't what you want to see. 782 00:37:37,450 --> 00:37:39,650 So feedback loops are super important. 783 00:37:40,050 --> 00:37:42,290 Another one that I want to highlight in this integrated 784 00:37:42,290 --> 00:37:46,180 auditing space that is. I get so excited about this. 785 00:37:46,260 --> 00:37:48,940 It's integrated planning. So you also mentioned, I keep 786 00:37:48,940 --> 00:37:51,180 anchoring back to this because the concepts that you're 787 00:37:51,180 --> 00:37:54,220 bringing up, the challenges that you've brought up, you are so 788 00:37:54,220 --> 00:37:56,860 not alone. I bet if you asked your audience 789 00:37:56,860 --> 00:37:59,220 today how many of them experienced some of these same 790 00:37:59,220 --> 00:38:01,700 challenges that you've experienced, most of them who 791 00:38:01,700 --> 00:38:04,180 have interacted with auditors have probably experienced that 792 00:38:04,180 --> 00:38:06,980 too. So we talked about getting an 793 00:38:06,980 --> 00:38:08,900 audit report. That doesn't help you. 794 00:38:09,140 --> 00:38:10,540 It's focused on the wrong things. 795 00:38:10,540 --> 00:38:13,690 It doesn't really add value. A great way to overcome that 796 00:38:13,690 --> 00:38:16,530 challenge is through integrated planning and that's where we're 797 00:38:16,530 --> 00:38:18,050 going to work super closely together. 798 00:38:18,050 --> 00:38:19,370 So Henry, I'm going to come audit you. 799 00:38:19,370 --> 00:38:21,970 We're going to work closely together and we're actually 800 00:38:21,970 --> 00:38:23,530 going to build out the audit scope together. 801 00:38:23,890 --> 00:38:26,570 So I'm going to be still independent because I get the 802 00:38:26,570 --> 00:38:29,350 final decision rights. If I say I want to look in this 803 00:38:29,350 --> 00:38:31,390 closet and you say, no, no, no, you don't need to look in that 804 00:38:31,390 --> 00:38:33,670 closet, but I still think I need to look in the closet, I'm going 805 00:38:33,670 --> 00:38:36,870 to look in the closet. But if you're also saying, hey, 806 00:38:36,910 --> 00:38:39,430 you know what, this is great. But what I'm really worried 807 00:38:39,430 --> 00:38:43,190 about or what I really need to go right is this area over here. 808 00:38:43,310 --> 00:38:45,710 Let's spend some time there and let's identify. 809 00:38:46,030 --> 00:38:48,230 So let's work together to identify what's most important 810 00:38:48,230 --> 00:38:50,270 to you. What can go wrong with that? 811 00:38:50,590 --> 00:38:52,190 Because I have my own ideas about it. 812 00:38:52,310 --> 00:38:55,260 Generally, they're aligned, but. It's so much more helpful when I 813 00:38:55,260 --> 00:38:58,460 get that confirmation from you. Or maybe you help me think about 814 00:38:58,460 --> 00:39:00,780 it in a different way. Like, yeah, that's really not 815 00:39:00,780 --> 00:39:02,900 when we go back to segregation of duties. 816 00:39:02,900 --> 00:39:05,500 You know, I may come in without integrated planning and say, 817 00:39:05,500 --> 00:39:07,860 like, give me those access lists and you're like, I mean, I could 818 00:39:07,860 --> 00:39:11,060 do that, but you help me understand, you know, I'm always 819 00:39:11,060 --> 00:39:13,620 thinking, well, we're looking for bad actors and that's the 820 00:39:13,620 --> 00:39:14,820 risk. And you're like, yeah, but 821 00:39:14,820 --> 00:39:18,180 actually mistakes happen more often than intentional bad code. 822 00:39:18,660 --> 00:39:21,340 So you can help me understand what those risks are and how 823 00:39:21,340 --> 00:39:24,140 they actually might manifest in your world. 824 00:39:24,500 --> 00:39:27,100 And then integrated planning, instead of me saying okay, I'm 825 00:39:27,100 --> 00:39:29,540 looking for a segregation of duties control, which doesn't 826 00:39:29,540 --> 00:39:32,180 exist because that's not the way you're doing things. 827 00:39:32,380 --> 00:39:35,060 You're going to tell me how you manage that risk and how you 828 00:39:35,060 --> 00:39:37,380 manage that risk. Let's say it is through those 829 00:39:37,380 --> 00:39:39,260 automated testing. Great. 830 00:39:39,660 --> 00:39:41,860 So now we've just saved ourselves a ton of time. 831 00:39:42,350 --> 00:39:44,590 Because now I understand what's really important to you. 832 00:39:44,910 --> 00:39:46,910 I understand those risks and what can go wrong. 833 00:39:46,910 --> 00:39:49,910 I understand how you control it. And then you can help me. 834 00:39:49,950 --> 00:39:51,350 You're saying, okay. You know what? 835 00:39:51,670 --> 00:39:54,630 Here's what I can provide you that will show you how those 836 00:39:54,630 --> 00:39:57,390 automated tests are set up. And then if you want to sit with 837 00:39:57,390 --> 00:40:01,150 me tomorrow, I can run through and I can send something through 838 00:40:01,150 --> 00:40:03,270 that's supposed to fail and send something through that's 839 00:40:03,270 --> 00:40:06,470 supposed to pass and we can get this test knocked out in a day. 840 00:40:06,950 --> 00:40:09,490 Great. Way better than going back and 841 00:40:09,490 --> 00:40:12,530 forth, getting confused, getting frustrated and hinting. 842 00:40:12,530 --> 00:40:14,850 You report that tells you you don't have duty segregated. 843 00:40:14,850 --> 00:40:16,610 You know that that was intentional. 844 00:40:16,850 --> 00:40:19,050 So I know I went on about that. This is something I'm super 845 00:40:19,050 --> 00:40:22,330 passionate about, but I really think that integrated planning 846 00:40:22,330 --> 00:40:25,410 in those feedback loops are something that audit clients can 847 00:40:25,410 --> 00:40:29,210 start doing today and really, really have a much better 848 00:40:29,210 --> 00:40:33,100 experience with their auditors. Thanks for sharing explicitly 849 00:40:33,100 --> 00:40:34,940 what happened in these kind of situations. 850 00:40:35,100 --> 00:40:37,620 I think it's always great to hear from the auditor's point of 851 00:40:37,620 --> 00:40:38,780 view. It's not just from client's 852 00:40:38,780 --> 00:40:40,780 point of view. And I like the quote that you 853 00:40:40,780 --> 00:40:43,180 mentioned earlier, right? Independence doesn't mean 854 00:40:43,180 --> 00:40:45,740 isolation, right. So that's integrate together, 855 00:40:46,020 --> 00:40:48,260 talk about the plan, the audit scope together. 856 00:40:48,580 --> 00:40:50,820 Like sometimes what happen is when we get audited, we just 857 00:40:50,820 --> 00:40:53,620 follow whatever scope they have, we wait for a couple of times, 858 00:40:53,620 --> 00:40:56,220 they go and ask us questions, we answer. 859 00:40:56,660 --> 00:40:58,700 Go back and forth and they'll come up with reports, right? 860 00:40:58,700 --> 00:41:00,980 So instead of doing that, I think we could do much better by 861 00:41:00,980 --> 00:41:03,460 doing this integrated auditing 2 point. 862 00:41:03,900 --> 00:41:05,420 Zero. I say real quick with that too 863 00:41:05,420 --> 00:41:07,580 is we're both aligned on the same goal. 864 00:41:07,580 --> 00:41:10,180 So I don't want to hand you an audit report that means nothing 865 00:41:10,180 --> 00:41:11,580 to you. You don't want to receive an 866 00:41:11,580 --> 00:41:13,220 audit report that means nothing to you. 867 00:41:13,260 --> 00:41:16,020 So it's not just helping you, it's helping the auditors too. 868 00:41:16,020 --> 00:41:18,020 And that's why it's super important for us to work 869 00:41:18,020 --> 00:41:21,380 together to make sure that we're both at the end producing this 870 00:41:21,380 --> 00:41:24,060 report that's going to add value to the organization. 871 00:41:24,540 --> 00:41:26,570 Right. And it will be best if both the 872 00:41:26,570 --> 00:41:29,930 clients and the auditors at the end actually like the reports 873 00:41:29,930 --> 00:41:31,610 that they produce. Yeah, they erase about it 874 00:41:31,610 --> 00:41:33,770 together just like what you did in the presentation. 875 00:41:33,970 --> 00:41:36,490 The other value that I think I'm very interested in is you 876 00:41:36,490 --> 00:41:39,610 mentioned about adaptable auditing in many of the audit 877 00:41:39,610 --> 00:41:42,410 process that we do actually is following some compliance 878 00:41:42,410 --> 00:41:45,210 framework certifications. And they do have a lot of 879 00:41:45,210 --> 00:41:48,090 checklists, a lot of areas, a lot of scope we mentioned. 880 00:41:48,370 --> 00:41:52,650 So how can we be more adaptable, like be flexible, know what to 881 00:41:52,650 --> 00:41:55,370 audit, when not to audit? So I think this is very 882 00:41:55,370 --> 00:41:58,410 interesting as well for people who normally go through audit by 883 00:41:58,410 --> 00:42:00,010 following compliance. Yeah. 884 00:42:00,010 --> 00:42:02,370 And part of that starts with the value driven. 885 00:42:02,370 --> 00:42:04,130 So focusing on what's most important. 886 00:42:04,450 --> 00:42:07,050 But then when it comes to the adaptability, so you have that 887 00:42:07,050 --> 00:42:10,610 stuff, how do you build in the ability to change and pivot, 888 00:42:11,050 --> 00:42:14,570 really prioritizing your work. So breaking the audit scope up 889 00:42:14,610 --> 00:42:17,930 into manageable pieces and limiting how much you're focused 890 00:42:17,930 --> 00:42:21,530 on at a time is something that really helps Dr. adaptability, 891 00:42:21,530 --> 00:42:24,250 and it's something that clients can influence as well. 892 00:42:24,950 --> 00:42:28,190 So instead of going in and saying we've got these 12 893 00:42:28,190 --> 00:42:30,390 controls that we're going to look at or these 12 compliance 894 00:42:30,390 --> 00:42:32,550 requirements we're going to look at and we start looking at all 895 00:42:32,550 --> 00:42:35,630 of them at once, we're going to figure out what is most 896 00:42:35,630 --> 00:42:38,190 important. So when it comes to compliance, 897 00:42:38,190 --> 00:42:41,390 certain things are going to have larger fines and larger impacts 898 00:42:41,390 --> 00:42:43,950 than others. If it's going to be a fine of a 899 00:42:43,950 --> 00:42:47,190 dollar every year, yeah, we want to comply with it, but. 900 00:42:47,530 --> 00:42:48,810 Do you need audit to tell you like this? 901 00:42:49,050 --> 00:42:50,730 That's not a good use of anybody's time. 902 00:42:50,770 --> 00:42:53,490 And I know I'm over exaggerating here, but bear with me. 903 00:42:53,770 --> 00:42:55,850 So if there's something that's going to cause millions of 904 00:42:55,850 --> 00:42:59,050 dollars in fines and a frequent basis and there's a decent 905 00:42:59,050 --> 00:43:01,130 chance of that, you want to focus on that. 906 00:43:01,490 --> 00:43:04,450 So prioritizing that instead of starting everything at once. 907 00:43:04,450 --> 00:43:06,410 So we're looking at something that's going to be a dollar and 908 00:43:06,410 --> 00:43:08,890 a fine and something that's going to be $1,000,000 and 909 00:43:08,890 --> 00:43:11,410 getting pieces of that and keeping those all going in 910 00:43:11,410 --> 00:43:13,010 process throughout the entire audit. 911 00:43:13,420 --> 00:43:16,620 You're going to limit what you're doing at a time and that 912 00:43:16,620 --> 00:43:18,820 is going to give you results sooner too. 913 00:43:19,180 --> 00:43:22,020 And then with those results we can pivot and say like have we 914 00:43:22,020 --> 00:43:24,500 done enough, have we audited enough, have we learned enough? 915 00:43:24,500 --> 00:43:28,500 So focusing on those areas that are most important, knocking 916 00:43:28,500 --> 00:43:30,940 those out first, and this is a concept, I mean a lot of these 917 00:43:30,940 --> 00:43:34,220 concepts should seem familiar. They should be things that. 918 00:43:34,650 --> 00:43:37,650 That you and your audience do already in your own daily work. 919 00:43:37,810 --> 00:43:40,890 So you're really well positioned to help your auditors pick these 920 00:43:40,890 --> 00:43:43,770 up and you can teach them. So limiting what we're doing at 921 00:43:43,770 --> 00:43:48,250 a time, picking it up, doing it, delivering it, starting another 922 00:43:48,250 --> 00:43:50,410 thing. And that helps with our clients 923 00:43:50,410 --> 00:43:53,010 too, because then you're not doing all that context 924 00:43:53,010 --> 00:43:54,570 switching. I'm not asking you about 925 00:43:54,570 --> 00:43:56,010 compliance with this piece of something. 926 00:43:56,010 --> 00:43:58,650 And then over here and then taking you back to that and 927 00:43:58,650 --> 00:44:01,830 where were we with that? So that's really something that 928 00:44:01,950 --> 00:44:03,750 can help us deliver those results sooner. 929 00:44:03,950 --> 00:44:06,710 And then we think through okay, now we've done these four 930 00:44:06,710 --> 00:44:09,030 things, do we need to keep going? 931 00:44:09,030 --> 00:44:11,830 What value will we get by completing the rest of this 932 00:44:11,830 --> 00:44:14,470 audit? If the answer is not that much 933 00:44:14,470 --> 00:44:17,030 and it's not worth it, then we stop because there could be 934 00:44:17,030 --> 00:44:19,110 something else like if we look at all these other things that 935 00:44:19,110 --> 00:44:20,830 we initially thought we were going to look at. 936 00:44:21,210 --> 00:44:23,050 We just have to do when everything's in process at the 937 00:44:23,050 --> 00:44:24,730 same time. So while we're limiting that, 938 00:44:24,730 --> 00:44:27,530 that gives us that opportunity to pause and think about should 939 00:44:27,530 --> 00:44:30,050 we spend our time collectively, everybody, not just the 940 00:44:30,050 --> 00:44:34,530 auditors, Should we spend all of our time finishing this or is 941 00:44:34,530 --> 00:44:36,850 there something else out there either in your space or a 942 00:44:36,850 --> 00:44:39,810 different space that is more important that we should pivot 943 00:44:39,810 --> 00:44:42,970 to. So I think that's a really good. 944 00:44:43,830 --> 00:44:46,790 Way to drive that response to change. 945 00:44:46,990 --> 00:44:49,110 And that's something that as a client you can help with that, 946 00:44:49,110 --> 00:44:50,910 help them, help your auditors with. 947 00:44:51,150 --> 00:44:53,430 You know what, I think you've provided us enough assurance. 948 00:44:53,430 --> 00:44:55,670 I think this is good. The value we're going to get out 949 00:44:55,670 --> 00:44:58,830 of this is minimal, like let's pivot to something else. 950 00:44:59,390 --> 00:45:01,950 Perfectly makes sense, right? Because I mean, in the tech 951 00:45:01,950 --> 00:45:04,590 world, we are so familiar with the HR concept, lean concept, 952 00:45:04,590 --> 00:45:06,190 right? So these things definitely make 953 00:45:06,190 --> 00:45:07,910 sense. And I think one more key from my 954 00:45:07,910 --> 00:45:10,910 point of view is also don't do this auditing when the certain 955 00:45:10,910 --> 00:45:12,830 time comes, right. So for example if you have a 956 00:45:12,870 --> 00:45:16,190 yearly requirements to do audit, then you only do that close to 957 00:45:16,190 --> 00:45:18,030 the time. So I think you can't do this 958 00:45:18,030 --> 00:45:20,190 definitely right because you have to complete all the 959 00:45:20,190 --> 00:45:23,030 checklist in one go. So I think maybe doing it also 960 00:45:23,070 --> 00:45:25,430 throughout the time in small iterations that they will value 961 00:45:25,430 --> 00:45:28,430 and pivot along the way. I think that may be also a great 962 00:45:28,430 --> 00:45:31,030 way to. Have this flexibility in terms 963 00:45:31,030 --> 00:45:33,150 of auditing. So we discussed a lot about 964 00:45:33,150 --> 00:45:36,110 internal auditing. So how do you see the external 965 00:45:36,110 --> 00:45:38,510 auditing part? Because these are different type 966 00:45:38,510 --> 00:45:41,310 of people, they may not come from the same organization, 967 00:45:41,510 --> 00:45:43,270 maybe the values may not be aligned. 968 00:45:43,310 --> 00:45:47,350 So is there any message that you wanna give for external auditors 969 00:45:47,350 --> 00:45:49,870 as well or for clients who are dealing with external auditors? 970 00:45:50,720 --> 00:45:52,480 Yeah. I would absolutely love to see 971 00:45:52,520 --> 00:45:55,760 external auditors leverage these practices too. 972 00:45:56,000 --> 00:45:58,960 So the book is focused on internal audit practices because 973 00:45:58,960 --> 00:46:02,840 that's where my background primarily is and there are 974 00:46:03,080 --> 00:46:06,520 different standards that external auditors are held to. 975 00:46:06,760 --> 00:46:09,960 And I'm not quite as familiar with those, but I would love to 976 00:46:09,960 --> 00:46:13,000 have a conversation about what are those requirements and how 977 00:46:13,000 --> 00:46:17,560 can external auditors also leverage these concepts so that 978 00:46:17,680 --> 00:46:20,380 they're also not feared. And so that they're better 979 00:46:20,380 --> 00:46:22,660 positioned. So by leveraging all of these 980 00:46:22,660 --> 00:46:26,580 concepts, driving by value, integrating into the client's 981 00:46:26,580 --> 00:46:30,060 work and being adaptable, those are going to set the auditors 982 00:46:30,100 --> 00:46:32,780 external or internal up for better success. 983 00:46:32,780 --> 00:46:34,900 Like I mentioned, we're going to be focusing on the right things. 984 00:46:34,900 --> 00:46:37,300 We're going to be not wasting our time or your time. 985 00:46:37,620 --> 00:46:39,900 Those are all things that the external auditors can benefit 986 00:46:39,900 --> 00:46:44,220 from as well. So absolutely I'd love to learn 987 00:46:44,220 --> 00:46:47,660 more about what those standards are that they're being held to 988 00:46:47,660 --> 00:46:49,380 and work with them to figure out how they. 989 00:46:49,500 --> 00:46:53,660 Can leverage these practices to not be feared, to have better 990 00:46:53,660 --> 00:46:56,300 working relationships while maintaining they have even more 991 00:46:56,300 --> 00:47:00,060 of an independence requirement than we do and you know, find 992 00:47:00,060 --> 00:47:02,700 efficiencies, add more value. I just think it would be great 993 00:47:02,700 --> 00:47:04,340 for them as well. Right. 994 00:47:04,740 --> 00:47:07,300 And it will be great if all conversation with auditors is 995 00:47:07,300 --> 00:47:09,700 like this, very friendly and we are collaborative. 996 00:47:09,700 --> 00:47:11,460 So I do hope we. Do things also. 997 00:47:11,460 --> 00:47:14,020 Happen. Yeah, in all the auditing 998 00:47:14,020 --> 00:47:15,580 experience that everyone is having. 999 00:47:16,000 --> 00:47:18,440 So thank you so much Clarissa, for explaining this concept. 1000 00:47:18,440 --> 00:47:21,840 Auditing with agility, I learned a lot and I probably have some 1001 00:47:21,840 --> 00:47:24,720 perspective change after you, you know, give some insights 1002 00:47:24,720 --> 00:47:26,400 about better practices for auditing. 1003 00:47:26,600 --> 00:47:29,320 So as we go to the end of our conversation, I have one last 1004 00:47:29,320 --> 00:47:32,000 thing that I would like to ask you, which I call 3 technical 1005 00:47:32,000 --> 00:47:34,400 leadership wisdom. Think of it just like advice 1006 00:47:34,400 --> 00:47:37,000 that you want to give to the listeners so that they can learn 1007 00:47:37,000 --> 00:47:38,880 from your expertise or your experience. 1008 00:47:39,000 --> 00:47:41,880 So would you be able to share the version of your 3 technical 1009 00:47:41,880 --> 00:47:45,520 leadership wisdom? I would love to so first. 1010 00:47:46,020 --> 00:47:47,700 Auditors are not your adversaries. 1011 00:47:48,060 --> 00:47:49,780 They should not be out to get you. 1012 00:47:50,340 --> 00:47:52,700 They actually should be a valuable resource for you to be 1013 00:47:52,700 --> 00:47:55,740 able to leverage. So absolutely, first thing I 1014 00:47:55,740 --> 00:47:58,820 want people to walk away with is that I want you to run to your 1015 00:47:58,820 --> 00:48:01,420 auditors, not from them. And I know that's going to take 1016 00:48:01,420 --> 00:48:03,540 some work, but keep that in mind. 1017 00:48:03,540 --> 00:48:05,660 We're not out to get you. We want to help you. 1018 00:48:06,340 --> 00:48:10,800 Second is, this is a journey. So it's not like laying a Scrum 1019 00:48:10,800 --> 00:48:14,440 framework onto an audit process. It's not a you know these five 1020 00:48:14,440 --> 00:48:17,720 steps and boom, you are agile. It's a journey. 1021 00:48:17,720 --> 00:48:20,160 And you and your audience probably know this from your own 1022 00:48:20,160 --> 00:48:23,200 experiences in these better ways of working. 1023 00:48:23,480 --> 00:48:26,760 So that's not an unfamiliar piece of wisdom to all of you. 1024 00:48:27,280 --> 00:48:29,840 But the way that you really get started on this journey is by 1025 00:48:29,840 --> 00:48:31,880 figuring out what's most important to you, you know 1026 00:48:31,880 --> 00:48:34,520 you're not going to apply everything in the book all at 1027 00:48:34,520 --> 00:48:36,440 once. You got to start small, so 1028 00:48:36,440 --> 00:48:38,360 figure out what is the most important to you. 1029 00:48:38,400 --> 00:48:40,840 Is it getting more value out of that audit? 1030 00:48:40,840 --> 00:48:43,120 Is it being able to respond to change? 1031 00:48:43,480 --> 00:48:45,920 I'd love to ask people, if you had a magic wand, how would you 1032 00:48:45,920 --> 00:48:49,280 use it to improve the audit process and what would that look 1033 00:48:49,280 --> 00:48:50,360 like? So that's what you're going to 1034 00:48:50,360 --> 00:48:54,600 start with. And 3rd, I would have each of 1035 00:48:54,600 --> 00:48:57,960 you reach out to your auditors today, connect with them. 1036 00:48:58,450 --> 00:49:01,690 So if you're in the middle of an audit, pause to go again. 1037 00:49:01,690 --> 00:49:04,530 Coffee is kind of my go to. It's like my peace offering. 1038 00:49:04,930 --> 00:49:07,090 Get coffee. Just set up a virtual chat, 1039 00:49:07,290 --> 00:49:10,410 start building it and strengthening that relationship, 1040 00:49:10,770 --> 00:49:13,890 and then provide them feedback. So we talked about feedback 1041 00:49:13,890 --> 00:49:15,450 loops. Activate a feedback loop. 1042 00:49:15,450 --> 00:49:18,010 So let them know. Now that you know, if you had 1043 00:49:18,010 --> 00:49:20,330 that magic wand of what it would look like, what's most important 1044 00:49:20,330 --> 00:49:21,930 to you? Tell your auditors. 1045 00:49:21,970 --> 00:49:24,290 Open that feedback loop. Ask how you can help. 1046 00:49:24,900 --> 00:49:27,260 And you know, I'm expanding like kind of into three, a 3B. 1047 00:49:27,540 --> 00:49:31,380 But I mentioned this earlier that many of you have a lot of 1048 00:49:31,380 --> 00:49:33,660 experience with these better ways of working and auditors 1049 00:49:33,660 --> 00:49:35,420 typically don't. This is something new for 1050 00:49:35,420 --> 00:49:38,860 auditors for a lot of us. So coach your auditors, tell 1051 00:49:38,860 --> 00:49:41,540 them, hey, you know what, I really think it'd be helpful if 1052 00:49:41,660 --> 00:49:44,180 you created a task board. I can show you how to do that 1053 00:49:44,180 --> 00:49:47,620 because we used JIRA and this is how it's been working for us or 1054 00:49:47,860 --> 00:49:50,840 maybe love standups. Hey, why don't you join some of 1055 00:49:50,840 --> 00:49:53,680 our daily standups and you can provide your audit status there 1056 00:49:53,680 --> 00:49:55,280 instead of having a separate meeting. 1057 00:49:55,280 --> 00:49:56,720 Or why don't you join into this meeting? 1058 00:49:56,720 --> 00:49:59,360 So coach your auditors. You're the experts in these 1059 00:49:59,360 --> 00:50:01,320 better ways of working. Teach them. 1060 00:50:01,360 --> 00:50:03,440 That's also going to help build those relationships and keep 1061 00:50:03,440 --> 00:50:07,160 those feedback loops going. So those are my 3 1/2 pieces of 1062 00:50:07,160 --> 00:50:09,990 wisdom. Okay, the third one maybe. 1063 00:50:09,990 --> 00:50:13,110 I would also call it auditors are human too, so maybe connect 1064 00:50:13,110 --> 00:50:15,390 with them right. Don't treat them as like robots 1065 00:50:15,390 --> 00:50:18,230 that just follow checklists. So they are humans as well and 1066 00:50:18,230 --> 00:50:21,470 they help us to get the same value, the same goal for the 1067 00:50:21,470 --> 00:50:23,830 organization. So thank you so much, Clarissa, 1068 00:50:23,830 --> 00:50:25,990 for this chat. So if people want to connect 1069 00:50:25,990 --> 00:50:28,630 with you or they want to ask you questions, is there a place 1070 00:50:28,630 --> 00:50:32,870 where they can find you online? Yep, so I would ask everyone to 1071 00:50:32,870 --> 00:50:35,630 check out my website clarissalucas.com. 1072 00:50:36,080 --> 00:50:39,520 I have a newsletter where I send out content that's helpful for 1073 00:50:39,520 --> 00:50:42,840 both auditors and audit clients, trying to help everyone have a 1074 00:50:42,840 --> 00:50:45,880 better audit experience. And then I'm also on LinkedIn as 1075 00:50:45,880 --> 00:50:47,680 well. And Henry, I really want to 1076 00:50:47,680 --> 00:50:49,120 thank you for having me here today. 1077 00:50:49,400 --> 00:50:52,600 Love getting in front of your type of audience so that people 1078 00:50:52,600 --> 00:50:55,360 don't have to fear their auditors, You know, I want 1079 00:50:55,440 --> 00:50:57,840 auditors and clients to get along and really, really 1080 00:50:57,840 --> 00:51:00,400 appreciate you having me on here and giving me the opportunity to 1081 00:51:00,400 --> 00:51:02,400 share. No worries so. 1082 00:51:02,970 --> 00:51:05,930 I am probably the first person who will not get scared to be 1083 00:51:05,930 --> 00:51:09,690 audited anymore, so I love it. So thank you for the insights 1084 00:51:09,690 --> 00:51:12,290 that you give in this episode. Clarissa, thank you so much for 1085 00:51:12,290 --> 00:51:14,490 the time again. So I hope people get 1086 00:51:14,530 --> 00:51:16,770 enthusiastic about their next audit experience. 1087 00:51:17,250 --> 00:51:20,330 And also for auditors, maybe you get a few lessons from here that 1088 00:51:20,330 --> 00:51:22,730 can change your practices. So thank you again for that. 1089 00:51:23,250 --> 00:51:28,170 My pleasure. Thank you for listening to this 1090 00:51:28,170 --> 00:51:31,570 episode and for staying right until the end if you highly 1091 00:51:31,570 --> 00:51:33,950 enjoyed it. I would appreciate if you share 1092 00:51:33,950 --> 00:51:36,270 it with your friends and colleagues who you think would 1093 00:51:36,270 --> 00:51:38,510 also benefit from listening to this episode. 1094 00:51:38,950 --> 00:51:41,710 And if you're new to the podcast, make sure to subscribe 1095 00:51:41,750 --> 00:51:44,150 and leave me your valuable review and feedback. 1096 00:51:44,470 --> 00:51:47,350 It helps me a lot in order to grow this podcast better. 1097 00:51:47,870 --> 00:51:50,750 You can also find the full show notes of this conversation on 1098 00:51:50,750 --> 00:51:53,710 the episode page at Technically journal dot dev website, 1099 00:51:54,030 --> 00:51:57,630 including the full transcript, interesting quotes and links to 1100 00:51:57,630 --> 00:52:00,030 the resources mentioned from the conversation. 1101 00:52:00,430 --> 00:52:02,830 And lastly. Make sure to subscribe to the 1102 00:52:02,830 --> 00:52:05,830 show's mailing list on techlyjuno dot dev to get 1103 00:52:05,830 --> 00:52:07,990 notified for any future episodes. 1104 00:52:08,510 --> 00:52:11,750 Stay tuned for the next Techlyjuno episode, and until 1105 00:52:11,750 --> 00:52:12,870 then, goodbye.