1 00:00:14,960 --> 00:00:17,240 Hello and welcome to the Behavioral Design Podcast. 2 00:00:17,720 --> 00:00:21,000 I'm Simon Schultzer, and today we have a very special episode 3 00:00:21,000 --> 00:00:24,840 where we're joined by Jared Peterson, a real scientist and 4 00:00:24,840 --> 00:00:28,960 colleague at Nuance Behavior. I have done all that I can to 5 00:00:28,960 --> 00:00:31,360 convince him to join because I think Jared is such a perfect 6 00:00:31,440 --> 00:00:34,040 person for the episode we have today. 7 00:00:34,600 --> 00:00:37,600 Jared is not only an expert in decision science, but he's 8 00:00:37,600 --> 00:00:41,040 recently also been involved in AI alignment work, which is 9 00:00:41,040 --> 00:00:44,440 perfect for today's discussion. He has also become a thought 10 00:00:44,440 --> 00:00:46,760 leader in the field with prominent writing including The 11 00:00:46,760 --> 00:00:51,080 Science of Context, and most recently, a fantastic review and 12 00:00:51,080 --> 00:00:55,000 ranking of all of the top payable science frameworks and 13 00:00:55,000 --> 00:00:57,960 models. So look that up. 14 00:00:58,000 --> 00:01:02,160 It's in the show notes. But All in all, Jared is the 15 00:01:02,160 --> 00:01:06,040 perfect person to help us make sense of the state of behavioral 16 00:01:06,040 --> 00:01:09,520 science, the state of AI, and everything in between. 17 00:01:09,520 --> 00:01:14,800 As we're today, we'll be trying to look back at some of the key 18 00:01:14,800 --> 00:01:18,840 breakthroughs in 2024 and peering into the future, trying 19 00:01:18,840 --> 00:01:23,280 to make sense of the state of our field and what we can expect 20 00:01:23,400 --> 00:01:27,840 in 2025. You can expect from us some hot 21 00:01:27,840 --> 00:01:31,680 takes, some predictions for the state of AI and payroll science, 22 00:01:32,120 --> 00:01:36,160 and maybe even some nuanced thinking and perspectives. 23 00:01:36,320 --> 00:01:40,000 We'll see. OK, let's dive in. 24 00:01:40,760 --> 00:01:42,200 Welcome, Jared. How are you? 25 00:01:43,680 --> 00:01:44,800 Thanks. I'm doing excellent. 26 00:01:44,840 --> 00:01:48,560 Our, I believe our takes should better be nuanced. 27 00:01:48,920 --> 00:01:52,480 That's kind of our label. Yeah, we we have to live up to 28 00:01:52,480 --> 00:01:53,160 that. We'll see. 29 00:01:53,200 --> 00:01:54,880 How are you feeling so far in 2025? 30 00:01:54,960 --> 00:01:59,840 What's your temperature? 2025, It'll be an interesting 31 00:01:59,840 --> 00:02:03,160 year, I am sure. I'm having a child this year, so 32 00:02:03,160 --> 00:02:06,360 that's the most prominent thing on my mind, but there's 33 00:02:06,360 --> 00:02:08,759 certainly some bigger things happening in the world as well. 34 00:02:08,759 --> 00:02:09,880 But it'll be an interesting year. 35 00:02:10,880 --> 00:02:13,720 Have you had that thought of like, what world am I bringing 36 00:02:13,760 --> 00:02:17,840 this child into? Quite a bit. 37 00:02:18,400 --> 00:02:22,240 I'm very curious how education because you know, for me, my 38 00:02:22,240 --> 00:02:26,400 childhood is pretty much defined by like the schools like that's 39 00:02:26,400 --> 00:02:29,280 what childhood was, is like you just go to school all the time 40 00:02:29,640 --> 00:02:31,920 and like those are my like most prominent memories. 41 00:02:32,200 --> 00:02:34,480 What is school going to look like in the age of AII? 42 00:02:35,160 --> 00:02:37,640 Have no idea. It'll be very interesting. 43 00:02:38,760 --> 00:02:43,200 Yeah, yeah, hopefully it doesn't have to be like Ready Player One 44 00:02:43,200 --> 00:02:47,440 or something like hopefully it actually it's going to be more 45 00:02:47,440 --> 00:02:50,080 like some form of utopian version. 46 00:02:50,240 --> 00:02:52,000 But I guess that's what we're going to dive into. 47 00:02:52,200 --> 00:02:56,520 But actually, before we look into this year, I kind of wanted 48 00:02:56,520 --> 00:03:00,160 to do something because obviously we're recording this 49 00:03:00,160 --> 00:03:05,400 in mid January and 2024 is already quickly becoming a blur. 50 00:03:05,840 --> 00:03:10,200 But I thought it would be useful to kind of first walk through 51 00:03:10,800 --> 00:03:14,880 what I guess I'll be sharing some things that I saw as key 52 00:03:14,880 --> 00:03:17,920 moments to reflect on from the past year. 53 00:03:18,400 --> 00:03:20,520 And I'll kind of throw them at you and see what you think. 54 00:03:20,520 --> 00:03:24,960 And I hope that kind of looking back will help us better looking 55 00:03:24,960 --> 00:03:27,760 forward and kind of forecasting as well what's going to happen 56 00:03:27,760 --> 00:03:31,120 this year. So Are you ready for my top 4 57 00:03:31,440 --> 00:03:34,760 happenings of 2024? Let's do it. 58 00:03:35,560 --> 00:03:42,000 OK, So for me, personal level, a big thing that happened was 59 00:03:42,000 --> 00:03:48,200 really experiencing and seeing the shifts from AI being 60 00:03:48,200 --> 00:03:51,320 something cool that you could chat with to AI becoming 61 00:03:51,320 --> 00:03:54,280 something multimodal in in a seamless way. 62 00:03:54,760 --> 00:03:58,080 And so this kind of multimodal evolution or revolution, 63 00:03:58,080 --> 00:04:00,200 whatever you call it, I feel like that was one of the big 64 00:04:00,200 --> 00:04:03,560 things that happened last year where most people started to 65 00:04:03,560 --> 00:04:06,880 notice that whatever interface they were using, suddenly they 66 00:04:06,880 --> 00:04:12,240 could generate a image from a prompt or a voice from a prompt 67 00:04:12,240 --> 00:04:15,640 or chat with a voice. It became such a kind of 68 00:04:15,640 --> 00:04:19,640 cohesive experience and accessible for most people to 69 00:04:19,640 --> 00:04:25,040 have that feeling of wow, I can talk to AII, can generate images 70 00:04:25,040 --> 00:04:26,920 with AII, can generate songs with AI. 71 00:04:26,920 --> 00:04:29,960 Whatever you you wanted to do, you could suddenly do it much 72 00:04:29,960 --> 00:04:33,000 more easily. So like quickly some things that 73 00:04:33,000 --> 00:04:37,040 popped out for me as like related to this open AI and the 74 00:04:37,040 --> 00:04:40,880 Scarlett Johansson lawsuit and eventually them launching 75 00:04:40,920 --> 00:04:43,240 advanced voice mode. I think that was a big thing. 76 00:04:43,240 --> 00:04:46,040 I look forward to it a long time being able to chat with an AI 77 00:04:46,040 --> 00:04:48,720 that it was like sounded like human and something that was a 78 00:04:48,720 --> 00:04:51,880 big, big moment. And it was delayed because they 79 00:04:51,880 --> 00:04:55,120 they realized they couldn't make it sound fully like Scarlett 80 00:04:55,120 --> 00:04:56,760 Johansson in What's the Film Again? 81 00:04:56,840 --> 00:04:58,720 Her. Her, yeah. 82 00:04:59,640 --> 00:05:01,960 So we didn't get a chance to have that full experience, but 83 00:05:01,960 --> 00:05:05,600 that was probably a good thing. I think that gave us a spice of 84 00:05:05,600 --> 00:05:09,720 like the arrogance that exists in a lot of this AI labs and 85 00:05:09,760 --> 00:05:11,760 that was interesting. Then we saw a mid journey 86 00:05:11,760 --> 00:05:15,920 releasing kind of what they now have a 6.1 model and a web 87 00:05:15,920 --> 00:05:18,160 interface for their model. Before you had to use the 88 00:05:18,240 --> 00:05:21,320 discord for mid journey, but like you can now generate like 89 00:05:21,320 --> 00:05:23,680 this amazing images. Like it's just insane what you 90 00:05:23,680 --> 00:05:27,880 can do and it's all now through an easy web interface. 11 labs 91 00:05:27,880 --> 00:05:29,600 made it possible to create voice clones. 92 00:05:30,000 --> 00:05:32,640 I cloned my voice, which was super surreal and weird. 93 00:05:33,280 --> 00:05:36,680 Hey, Jen and Synthesia allowed for video clones, which I also 94 00:05:36,680 --> 00:05:38,400 did, which was even more surreal. 95 00:05:38,560 --> 00:05:41,200 And then on top of that, there was all of these tools. 96 00:05:41,200 --> 00:05:45,240 They can send out kind of clones of you to call for you and make 97 00:05:45,240 --> 00:05:49,480 voice clone like voice clone calls for you, which was like 98 00:05:49,480 --> 00:05:52,840 super strange. And maybe the last thing that 99 00:05:52,840 --> 00:05:56,720 happened around this was notebook LM was was one of the 100 00:05:56,720 --> 00:05:58,120 tools that came out towards the end of the year. 101 00:05:58,520 --> 00:06:00,960 And they launched this kind of podcast mode where you can 102 00:06:00,960 --> 00:06:04,080 basically feed your own documents and then have two AI 103 00:06:04,080 --> 00:06:08,880 podcast hosts discussing what you gave them as like they were 104 00:06:08,880 --> 00:06:11,360 kind of having a shot and it sounded super realistic. 105 00:06:11,800 --> 00:06:15,200 So yeah, that's the multimodal evolution revolution. 106 00:06:15,240 --> 00:06:16,480 What's your thoughts on this here? 107 00:06:17,560 --> 00:06:19,600 I feel like I've barely touched it. 108 00:06:19,840 --> 00:06:22,680 There's so much going on that it's so hard to keep up with 109 00:06:22,680 --> 00:06:23,760 everything. I've. 110 00:06:23,800 --> 00:06:27,000 I've done a few things. I remember my first kind of wow 111 00:06:27,000 --> 00:06:32,200 moment with the multimodal was I was struggling in Framer, which 112 00:06:32,200 --> 00:06:35,000 is the software we use for developing, like my personal 113 00:06:35,000 --> 00:06:37,400 website and also the nuanced behavior website. 114 00:06:37,920 --> 00:06:39,560 And I couldn't figure this thing out. 115 00:06:39,880 --> 00:06:44,120 And I took a screenshot of the page and I was like, all the 116 00:06:44,120 --> 00:06:47,240 instructions online are telling me to do one thing, but clearly 117 00:06:47,240 --> 00:06:50,120 the interface has changed and that's not working. 118 00:06:50,880 --> 00:06:53,280 So here's a screenshot. Tell me where to click. 119 00:06:53,880 --> 00:06:56,720 And it told me specifically where I needed to click in that 120 00:06:56,720 --> 00:07:00,000 image. And it was like, oh, wow, that's 121 00:07:00,000 --> 00:07:02,120 pretty cool. Because like, that's the thing 122 00:07:02,120 --> 00:07:05,200 that, you know, if you have like parents or grandparents that 123 00:07:05,200 --> 00:07:08,120 aren't as tech savvy and you have to like guide them to like, 124 00:07:08,160 --> 00:07:10,720 OK, like here's like physically where you're going to click. 125 00:07:11,240 --> 00:07:12,920 That's what chat CBT was doing for me. 126 00:07:14,360 --> 00:07:16,960 That kind of creeped me out because that's some really 127 00:07:16,960 --> 00:07:20,600 advanced image recognition, and then being able to translate 128 00:07:20,680 --> 00:07:23,280 what it's seeing in the image to actual text, it's pretty 129 00:07:23,280 --> 00:07:26,200 impressive. Yeah, that's a great example. 130 00:07:26,320 --> 00:07:29,320 And I can relate to having for many years been kind of the form 131 00:07:29,320 --> 00:07:33,200 of tech support for several family members. 132 00:07:33,200 --> 00:07:37,880 So, but yeah, that is wild. I agree with you and I actually 133 00:07:37,880 --> 00:07:42,880 picked up tweet or I think it was maybe posted on on Facebook 134 00:07:43,200 --> 00:07:46,480 used the other day by do you know Paul Schrader? 135 00:07:46,680 --> 00:07:49,360 He he wrote taxi driver and regime bowl. 136 00:07:49,360 --> 00:07:53,400 Do you know him, the writer? No, So he's like a very, very 137 00:07:53,400 --> 00:07:58,080 significant screenwriter. And he had one of those moments 138 00:07:58,080 --> 00:08:01,560 that, like, similar to you, he had to vent online. 139 00:08:01,560 --> 00:08:04,800 And he said, I've just come to realize that AI is smarter than 140 00:08:04,800 --> 00:08:08,600 I am, has better ideas, has more efficient ways to execute them. 141 00:08:09,120 --> 00:08:12,240 This is an extensional moment aching to what Caspero felt in 142 00:08:12,240 --> 00:08:15,720 1997 when he realized Deep Blue was going to beat him at chess. 143 00:08:15,880 --> 00:08:19,200 As I just sent Chad GPTA script I'd written a few years ago and 144 00:08:19,200 --> 00:08:21,520 asked for improvements. And in five seconds it responded 145 00:08:21,520 --> 00:08:25,080 with notes that was good or better than I've received from a 146 00:08:25,080 --> 00:08:27,680 film executive or a fellow writer. 147 00:08:29,120 --> 00:08:35,240 I'm done basically. So that is like someone sharing 148 00:08:35,240 --> 00:08:37,679 something and then having that kind of interaction with AI in 149 00:08:37,679 --> 00:08:39,760 some form of semi multimodal way. 150 00:08:40,760 --> 00:08:45,160 Though to be fair, I think chachibti is best when you 151 00:08:45,160 --> 00:08:48,200 already have some kind of base for it to build on. 152 00:08:48,680 --> 00:08:53,560 If you say start from scratch, sometimes it's not quite there 153 00:08:53,880 --> 00:08:57,160 because you kind of want to embed in whatever your human 154 00:08:57,160 --> 00:09:00,160 insights are. So if you're making a movie and 155 00:09:00,160 --> 00:09:03,080 you have some kind of theme that you're trying to get through, if 156 00:09:03,080 --> 00:09:07,400 you are the one designing context around that theme, then 157 00:09:07,400 --> 00:09:09,080 it can really help you perfect that. 158 00:09:09,840 --> 00:09:12,200 But is it really going to do it from scratch? 159 00:09:12,320 --> 00:09:15,320 That I'm a little bit more skeptical about and, and most of 160 00:09:15,320 --> 00:09:20,040 my use are with AI is more about cleaning up what I already have. 161 00:09:20,320 --> 00:09:23,040 I'll do some kind of stream of consciousness and it'll clean it 162 00:09:23,040 --> 00:09:26,920 up rather than here's the like, rather than just saying, hey, 163 00:09:26,920 --> 00:09:29,800 write an article or here's the germ of an idea, write the 164 00:09:29,800 --> 00:09:32,520 article. No, first I have to do some work 165 00:09:32,800 --> 00:09:34,200 and then it's better at cleaning up. 166 00:09:35,480 --> 00:09:39,160 Yeah, that's great take. And I will say that probably the 167 00:09:39,160 --> 00:09:41,880 thing I've said the most to people as like an advice of sort 168 00:09:41,880 --> 00:09:45,800 is like leverage your expertise. Like if you have expertise that 169 00:09:45,800 --> 00:09:48,960 is super, super valuable with AI, because if you have no 170 00:09:48,960 --> 00:09:50,760 expertise and you ask for something, you're going to get 171 00:09:50,760 --> 00:09:52,600 something very generic for the most part. 172 00:09:53,120 --> 00:09:56,480 And you'll have, even if you get something really good, there 173 00:09:56,480 --> 00:09:59,160 will be something there that maybe is terrible and you won't 174 00:09:59,160 --> 00:10:01,640 be able to really pick up on it. So having expertise and 175 00:10:01,640 --> 00:10:04,720 leveraging that is is really powerful and that takes us to 176 00:10:06,200 --> 00:10:09,240 something interesting like this kind of leads us into another 177 00:10:09,240 --> 00:10:12,160 thing that happened last year, which was surpassing human 178 00:10:12,160 --> 00:10:15,960 benchmarks. And that was kind of these AI 179 00:10:15,960 --> 00:10:17,520 models, especially in the frontier models. 180 00:10:17,520 --> 00:10:21,760 So the ones by open AI or entropic or even some of the 181 00:10:21,840 --> 00:10:27,960 open source one, they started to perform as well or outperforming 182 00:10:27,960 --> 00:10:31,960 humans in many tasks and at some tasks outperforming human 183 00:10:31,960 --> 00:10:34,800 experts or PhD level performance. 184 00:10:35,320 --> 00:10:41,480 And so for example, it started with ChatGPT or like GPT 4 185 00:10:41,880 --> 00:10:50,080 passing the bar exam, then more advanced version, I believe it 186 00:10:50,080 --> 00:10:59,960 was O one or it was 4 O that was able to basically perform as 187 00:10:59,960 --> 00:11:06,040 good or better when it came to some creative thinking and 188 00:11:07,160 --> 00:11:11,560 divergent thinking tasks. So it was a nature study around 189 00:11:11,560 --> 00:11:15,440 this where a were shown to be more creative than humans on 190 00:11:15,440 --> 00:11:18,600 divergent thinking tasks. So basically creating multiple 191 00:11:18,600 --> 00:11:21,800 unique ideas or solutions to a problem and trying to solve it. 192 00:11:23,120 --> 00:11:27,440 There was one where AI generated poetry was indistinguishable 193 00:11:27,440 --> 00:11:31,080 from human written poetry, which is kind of probably where you 194 00:11:31,080 --> 00:11:33,760 would expect AI to be really strongly performing. 195 00:11:33,800 --> 00:11:37,200 It was kind of more of a less of a study, but more of a 196 00:11:37,560 --> 00:11:42,840 experiment of sort where 1278 people who said that they hated 197 00:11:42,880 --> 00:11:45,760 AI art were unable to distinguish between AI art and 198 00:11:45,760 --> 00:11:51,560 human art. There were quite a lot of strong 199 00:11:51,560 --> 00:11:56,320 benchmarks from the one model from Open AI on various things 200 00:11:56,440 --> 00:12:01,720 from math to legal analysis to various things. 201 00:12:01,720 --> 00:12:05,680 It was pushing quite close to biology, science related topics, 202 00:12:05,680 --> 00:12:07,480 pushing quite close to PhD level. 203 00:12:07,920 --> 00:12:12,520 And then we ended the year with this release of the report 204 00:12:12,760 --> 00:12:17,000 saying that Open AI SO3 model that has not been released yet, 205 00:12:17,480 --> 00:12:21,320 but has been used to basically pass what's called the ARC ADI 206 00:12:21,400 --> 00:12:25,440 benchmark and which is a benchmark to try to kind of 207 00:12:25,440 --> 00:12:29,840 approximate for some level of artificial general intelligence. 208 00:12:30,280 --> 00:12:36,320 And again, maybe with some arrogance, Open AI came out and 209 00:12:36,320 --> 00:12:38,400 basically alluded to that fact that like, yeah, we've kind of 210 00:12:38,800 --> 00:12:42,280 cracked the code in some ways. So what do you make make of 211 00:12:42,280 --> 00:12:45,240 these kind of what I would call like the vibe shift in some ways 212 00:12:45,240 --> 00:12:47,280 in the talk around AI performance? 213 00:12:48,560 --> 00:12:52,240 It's really hard to know how important any given metric or 214 00:12:52,240 --> 00:12:55,920 benchmark is because of essentially Goodheart's Law. 215 00:12:56,600 --> 00:13:00,120 Goodheart's Law means that once a measure becomes a target, it 216 00:13:00,120 --> 00:13:04,200 ceases to be a good measure. And in these cases, well, it's 217 00:13:04,200 --> 00:13:10,560 like there are so many test studies of the bar exam online 218 00:13:11,200 --> 00:13:13,440 that of course AI is going to know how to do that. 219 00:13:13,960 --> 00:13:16,680 And as soon as the AI companies are like, oh, here's the new 220 00:13:16,680 --> 00:13:20,000 benchmark, well, they publish it and then they train the model to 221 00:13:20,000 --> 00:13:23,040 beat that benchmark. And so it's really hard to know 222 00:13:23,040 --> 00:13:29,600 is, are these fundamental leaps in performance or is this a very 223 00:13:29,600 --> 00:13:33,320 narrow, all right, we've made our AI just a little more 224 00:13:33,320 --> 00:13:36,640 general, a little better able to do this one specific task. 225 00:13:36,920 --> 00:13:40,040 And that's still impressive. That's still a cool thing, but 226 00:13:40,040 --> 00:13:43,280 it's not clear that it's as impressive as it sounds on the 227 00:13:43,280 --> 00:13:46,560 surface. Yeah, I think you hit the nail 228 00:13:46,560 --> 00:13:49,280 on the head. It can be really impressive on 229 00:13:49,280 --> 00:13:53,240 one level, like doing something again, like trying to pass the 230 00:13:53,240 --> 00:13:55,400 bar exam because again, it has a lot of training data on that. 231 00:13:55,760 --> 00:13:59,800 But then you give it something relatively easy that any human 232 00:13:59,800 --> 00:14:01,760 would be quite easy able to make sense of. 233 00:14:01,760 --> 00:14:05,280 Like what's it called? Like this game of tic tac toe or 234 00:14:05,280 --> 00:14:08,480 something like some from a relatively easy challenge, 235 00:14:08,920 --> 00:14:12,720 common task fails completely. So we're still at a point where 236 00:14:12,720 --> 00:14:15,600 it's in person. I guess what I would ask to kind 237 00:14:15,600 --> 00:14:18,520 of change the question here a little bit, what from last year 238 00:14:18,520 --> 00:14:21,160 what, what experience impressed you the most with AI? 239 00:14:21,160 --> 00:14:26,800 Like was there some form of specific use case or any general 240 00:14:26,800 --> 00:14:28,960 thing that AI was able to accomplish last year that you 241 00:14:28,960 --> 00:14:32,600 were impressed with? There's quite a few things I was 242 00:14:32,600 --> 00:14:37,000 really impressed with Notebook LM, the podcast feature. 243 00:14:37,000 --> 00:14:41,200 So basically in Notebook LM, what you do is you upload one 244 00:14:41,200 --> 00:14:44,000 paper or two papers. You know, I, I sometimes upload 245 00:14:44,000 --> 00:14:49,040 3 or 4 academic papers and then I ask for summaries and it'll 246 00:14:49,400 --> 00:14:53,080 very closely reference the source material. 247 00:14:53,520 --> 00:14:57,120 So we're getting away from the hallucinations, which I think 248 00:14:57,120 --> 00:15:00,920 was just like a major roadblock for trusting LMSLMS. 249 00:15:01,200 --> 00:15:05,680 And now that I can see, OK, here's what the LLM is saying 250 00:15:06,040 --> 00:15:09,840 and here's why, because it'll have like a footnote and I can 251 00:15:09,840 --> 00:15:12,880 reference the actual text from the original source materials. 252 00:15:13,160 --> 00:15:16,760 So that was really impressive. The podcast you can't do that. 253 00:15:16,760 --> 00:15:19,160 You can't see why it's saying what it's saying. 254 00:15:19,360 --> 00:15:23,240 So I'm, I, I trust it less. I'm I'm afraid that it's 255 00:15:23,240 --> 00:15:25,920 hallucinating because I can't verify that it's not. 256 00:15:26,520 --> 00:15:28,240 But I think that was a major breakthrough. 257 00:15:28,960 --> 00:15:33,320 And another major thing that really impressed me was my wife 258 00:15:33,320 --> 00:15:37,840 asked me for something in her in the bathroom and I opened this 259 00:15:37,840 --> 00:15:42,600 closet and there's 2 billion things of makeup and lotions 260 00:15:42,600 --> 00:15:44,480 and, and whatever else. And I have no idea what any of 261 00:15:44,480 --> 00:15:46,200 it is. And the thing she asked for, I 262 00:15:46,200 --> 00:15:48,960 have no idea, like what color it is. 263 00:15:48,960 --> 00:15:52,520 I don't know the brand. And so I just say, I open up 264 00:15:52,520 --> 00:15:56,360 ChatGPT and I say, hey, ChatGPT, you know, look inside this 265 00:15:56,360 --> 00:15:58,520 closet and see if you can find this thing for me. 266 00:15:58,800 --> 00:16:03,080 And I turn on the camera and scan and it finds it and it's 267 00:16:03,080 --> 00:16:07,040 like, Oh my gosh. Because like it had additional 268 00:16:07,040 --> 00:16:09,560 context that I didn't in that situation, which is like, you 269 00:16:09,560 --> 00:16:10,960 know, it knew what I was looking for. 270 00:16:10,960 --> 00:16:14,360 It knew the likely brands and therefore it knew what colours 271 00:16:14,360 --> 00:16:16,840 it was looking for. And I didn't have that context. 272 00:16:17,320 --> 00:16:21,040 Now most of the time I have way more context than Chachi PT, 273 00:16:21,200 --> 00:16:23,440 which is why I can do things that it can't. 274 00:16:23,680 --> 00:16:26,560 You know, tic tac toe, it sucks at it because it doesn't, you 275 00:16:26,560 --> 00:16:30,040 know, it can't even remember where the XS and OS are. 276 00:16:30,320 --> 00:16:33,640 And so I have this context, but we're getting to a point where 277 00:16:33,640 --> 00:16:37,120 now it has more context than I do, and it makes it more 278 00:16:37,120 --> 00:16:42,480 impressive than me at times. And you're staying with that. 279 00:16:44,280 --> 00:16:49,680 I think we're probably starting to feel like at some moment we 280 00:16:49,680 --> 00:16:52,360 feel a little like Paul Trader in terms of like we feel a 281 00:16:52,360 --> 00:16:56,120 little outdone by AI. How are you feeling about that 282 00:16:56,520 --> 00:16:58,320 sitting without feeling like, is that something that you're 283 00:16:58,320 --> 00:17:00,440 excited about or is it uncomfortable or how do you feel 284 00:17:00,440 --> 00:17:05,880 about that? I, I wish I had an answer. 285 00:17:06,240 --> 00:17:09,520 We've talked about this a bit online, but for me it's a bit of 286 00:17:09,520 --> 00:17:12,599 a cycle. Sometimes I'm really excited and 287 00:17:12,599 --> 00:17:14,240 it's like, wow, this is just so impressive. 288 00:17:14,240 --> 00:17:17,160 And other times it's like, man, this is kind of terrifying. 289 00:17:17,599 --> 00:17:20,079 And sometimes I'm like, this is, it's so good. 290 00:17:20,079 --> 00:17:22,560 And other times I'm like, well, it's not bad good, You know, 291 00:17:22,560 --> 00:17:24,280 look at these silly mistakes it's making. 292 00:17:24,680 --> 00:17:26,720 And I'm kind of all over the place and it kind of just 293 00:17:26,720 --> 00:17:30,160 depends like minute to minute, you know what, what particular 294 00:17:30,160 --> 00:17:33,280 problem I'm working on. As you alluded to in the intro, 295 00:17:33,280 --> 00:17:35,440 I'm working on a couple of AI alignment projects. 296 00:17:35,880 --> 00:17:39,400 And sometimes the AI is just doing really stupid things. 297 00:17:39,400 --> 00:17:43,520 You know, one of the projects is about triage and like the AI is 298 00:17:43,520 --> 00:17:47,320 like, well, what if I put a bandage or a tourniquet around 299 00:17:47,320 --> 00:17:50,760 this guy's neck? It's like, no, that would choke 300 00:17:50,760 --> 00:17:52,040 him out. That would kill him. 301 00:17:52,040 --> 00:17:55,040 Do not put a a tourniquet around this guy's neck. 302 00:17:55,400 --> 00:18:00,720 And it's so stupid at times, but at the same time so impressive. 303 00:18:01,080 --> 00:18:04,240 And yeah, it's so unclear. Is this something that's going 304 00:18:04,240 --> 00:18:07,080 to be able to do our jobs in a few years? 305 00:18:07,880 --> 00:18:09,840 It's it's so hard to predict something like that. 306 00:18:11,320 --> 00:18:14,520 Yeah, but so we shall very soon get ready. 307 00:18:15,160 --> 00:18:18,280 But yeah, I'll just quickly say two things. 308 00:18:18,280 --> 00:18:24,000 So I, I think it is something that we probably should start to 309 00:18:24,440 --> 00:18:27,680 sit with a little bit more that feeling of like there's nothing 310 00:18:27,680 --> 00:18:31,160 that says that at some point, especially when it comes to 311 00:18:31,840 --> 00:18:35,960 specific knowledge or tasks or someone that AI is likely not 312 00:18:35,960 --> 00:18:37,960 going to be like more capable of specific things. 313 00:18:38,440 --> 00:18:40,520 And so you're going to practice something this year. 314 00:18:40,520 --> 00:18:42,320 It's probably acceptance. It's a good thing to practice 315 00:18:42,320 --> 00:18:47,120 this year accepting that kind of fact and then take me like 316 00:18:47,120 --> 00:18:51,560 quickly around a notebook. LM I used it as a fun thing over 317 00:18:51,560 --> 00:18:56,720 Christmas because I picked up a lot of reading and I ended up 318 00:18:56,720 --> 00:18:58,920 like while I was reading, putting in some notes and some 319 00:18:58,920 --> 00:19:02,280 voice notes in my own kind of AI note taking app. 320 00:19:02,280 --> 00:19:05,040 I'm sorry. And then afterwards I kind of 321 00:19:05,280 --> 00:19:07,800 have this function where basically when I tell it to like 322 00:19:07,800 --> 00:19:11,040 summarizes things in a format that I like in the end. 323 00:19:11,560 --> 00:19:14,600 And usually I used to have it as that because it like gives me 324 00:19:14,600 --> 00:19:17,840 kind of, OK, these were these were my thoughts from this book. 325 00:19:18,280 --> 00:19:21,960 But then I also put that into notebook LM and I created like 326 00:19:21,960 --> 00:19:25,600 this podcast, as you alluded to, and it was really fun because 327 00:19:25,600 --> 00:19:29,000 then it was this weekend. I was like, wait, I read this 328 00:19:29,000 --> 00:19:30,200 book. Like, what was my thoughts? 329 00:19:30,200 --> 00:19:31,400 Like what was this book about again? 330 00:19:31,760 --> 00:19:34,640 And then I listened to this podcast that I had created back 331 00:19:34,640 --> 00:19:37,960 then and it was like perfect because it was like a 1520 332 00:19:37,960 --> 00:19:42,160 minute podcast talking about all of the points that I had to 333 00:19:42,160 --> 00:19:45,120 explore, but also some further exploration around those points 334 00:19:45,120 --> 00:19:47,880 and summarize some of the stuff that happened in the book. 335 00:19:47,880 --> 00:19:53,160 So that'd really help with my kind of retrieval and a memory 336 00:19:53,160 --> 00:19:57,640 of the book and helping me kind of better, I guess, absorb it 337 00:19:57,640 --> 00:20:00,480 long term and, and thinking more about it in hindsight. 338 00:20:00,880 --> 00:20:05,520 So yeah, definitely recommend everyone to use notebook LM 339 00:20:05,520 --> 00:20:08,160 because it's free so far and it's, it's very accessible. 340 00:20:08,200 --> 00:20:10,560 So it's useful. Great. 341 00:20:10,680 --> 00:20:13,360 And there you go, a free recommendation for a tool. 342 00:20:14,160 --> 00:20:15,000 Yeah. Any quick thing? 343 00:20:16,440 --> 00:20:19,760 I was going to mention what's interesting about Notebook LM is 344 00:20:19,760 --> 00:20:24,360 that it is so tied to the source material that I sometimes don't 345 00:20:24,360 --> 00:20:28,120 like using it because sometimes I want to generate new and novel 346 00:20:28,120 --> 00:20:31,680 insights and it's so tied to the source materials that it can't. 347 00:20:32,520 --> 00:20:35,400 So I actually, it's still useful to kind of switch back and forth 348 00:20:35,400 --> 00:20:38,480 between something that hallucinates a bit and you have 349 00:20:38,480 --> 00:20:41,520 that risk because that hallucination still might be 350 00:20:41,520 --> 00:20:44,920 useful. And I can still validate that by 351 00:20:44,920 --> 00:20:46,320 going back to the source materials. 352 00:20:46,600 --> 00:20:50,320 And so I, I think there's sort of a pipeline where you want 353 00:20:50,320 --> 00:20:53,160 something maybe in the beginning that's really tied to the source 354 00:20:53,160 --> 00:20:55,720 materials. But at some point when you have 355 00:20:55,720 --> 00:20:57,880 a little bit more expertise and you really understand the area, 356 00:20:58,200 --> 00:21:01,480 then you want to kind of turn up the temperature, so to speak, 357 00:21:01,480 --> 00:21:04,000 and and deal with something that's a little bit more 358 00:21:04,000 --> 00:21:07,360 creative. Yeah, no, I, I've noticed that 359 00:21:07,360 --> 00:21:08,360 too. And I think like you said, I 360 00:21:08,360 --> 00:21:11,520 think they have twist the temperature to be extremely like 361 00:21:11,520 --> 00:21:14,120 basically, so it's not hallucinating anything at all. 362 00:21:15,200 --> 00:21:17,920 And usually for a lot of stuff like the sweet spot is around 363 00:21:17,920 --> 00:21:20,640 like .7 or something like you want to have a little bit of 364 00:21:20,640 --> 00:21:24,160 like hallucination or like allowing it to be creative. 365 00:21:25,000 --> 00:21:29,040 But yeah, that's a great take and helps us now also to I think 366 00:21:29,040 --> 00:21:31,320 move to. I was going to talk about these 367 00:21:31,320 --> 00:21:36,560 AS22 events, but basically it's one is AI agents and and the 368 00:21:36,560 --> 00:21:38,680 second one was going to be like the dead Internet, like the 369 00:21:38,680 --> 00:21:41,440 flood of synthetic content. And I think they're kind of one 370 00:21:41,440 --> 00:21:44,680 of the same in some ways. So quickly the defining AI 371 00:21:44,680 --> 00:21:47,320 agents here. So what I saw and experienced 372 00:21:47,320 --> 00:21:53,440 last year was kind of the thing that shifted was that suddenly 373 00:21:55,160 --> 00:22:00,080 not only could you use API directly with the model in some 374 00:22:00,080 --> 00:22:03,640 form of shot interface or or similar versions of using it, 375 00:22:04,120 --> 00:22:07,440 but there were more and more frameworks and more and more way 376 00:22:07,800 --> 00:22:13,040 to basically allow some form of autonomy and give them whatever 377 00:22:13,040 --> 00:22:15,640 language model that you want to use, give it basic access to 378 00:22:15,640 --> 00:22:18,200 certain tasks. So initially it was like super 379 00:22:18,200 --> 00:22:24,480 simple stuff like you can create some form of your own agents 380 00:22:24,480 --> 00:22:28,920 that can send an e-mail for you or that can book a calendar 381 00:22:28,920 --> 00:22:31,960 appointment for you or do some of those like quite simple 382 00:22:31,960 --> 00:22:35,000 things. And then technically, as I 383 00:22:35,000 --> 00:22:37,720 experiment, experiment with this, you can then like add more 384 00:22:37,720 --> 00:22:41,800 and more tools or tasks that this can do. 385 00:22:41,800 --> 00:22:45,320 And it can kind of like based on what you give it as a prompt or 386 00:22:45,320 --> 00:22:50,000 ask, it will has autonomy around what it chooses to use and and 387 00:22:50,080 --> 00:22:52,560 what combination of tools they might use in order to accomplish 388 00:22:52,560 --> 00:22:56,880 the task that you want it to do. And so that is I think something 389 00:22:57,160 --> 00:23:03,200 that is exciting because it's allows for a lot of potential 390 00:23:03,200 --> 00:23:06,320 use cases that it's not possible use with the kind of a one to 391 00:23:06,320 --> 00:23:12,920 one exchange of sorts. And it started to have some form 392 00:23:12,920 --> 00:23:18,920 of call it implications already last year where there was a lot 393 00:23:18,920 --> 00:23:20,400 of frameworks. Again, as I mentioned that came 394 00:23:20,400 --> 00:23:23,800 out both how to use this in automation platforms and also 395 00:23:23,800 --> 00:23:28,360 like how to use some form of Python based or other type of 396 00:23:28,360 --> 00:23:32,840 code based frameworks for this. And yeah, I'll stop there. 397 00:23:33,720 --> 00:23:36,360 Jared, did you notice anything around this or any any thoughts 398 00:23:36,360 --> 00:23:40,560 on aged agent stuff? I'm still terrified to use an AI 399 00:23:40,560 --> 00:23:43,080 agent. I just don't quite trust it 400 00:23:43,160 --> 00:23:46,240 enough. I think there's probably a few 401 00:23:46,240 --> 00:23:48,960 things in my life that it might be able to do. 402 00:23:49,360 --> 00:23:52,600 Like, I don't know, maybe there's something around my file 403 00:23:52,600 --> 00:23:55,520 system that it could clean it up a little bit more quickly than I 404 00:23:55,520 --> 00:23:58,160 could. But even that it's like, well, 405 00:23:59,520 --> 00:24:02,000 in some ways, the file system, my computer is, you know, a 406 00:24:02,000 --> 00:24:05,000 second brain for me and like, I need to know exactly how 407 00:24:05,000 --> 00:24:08,720 everything is organized. So what what can I actually 408 00:24:08,720 --> 00:24:12,760 trust an AI agent to do? I I'm just not sure yet. 409 00:24:12,760 --> 00:24:14,720 I haven't found the use case where I'm like, oh, this would 410 00:24:14,720 --> 00:24:17,840 be really cool and I would really trust an AI to do this. 411 00:24:18,200 --> 00:24:21,440 I'm always just a little skeptical, a little hesitant. 412 00:24:21,440 --> 00:24:22,960 That's. Very understandable though, I 413 00:24:22,960 --> 00:24:25,200 think a lot of people can feel what you described. 414 00:24:25,560 --> 00:24:28,880 And have you noticed there was actually something papers that 415 00:24:28,880 --> 00:24:32,120 came out, especially the end of last year using agents in 416 00:24:32,120 --> 00:24:34,520 research? There's one paper at Stanford 417 00:24:34,520 --> 00:24:38,320 and Google study, I think it was literally called generative AI 418 00:24:38,320 --> 00:24:40,520 simulation of 1000 people. Did you see that paper? 419 00:24:40,760 --> 00:24:44,800 I haven't, but I I assume it's similar to what other people 420 00:24:44,800 --> 00:24:47,920 have been doing, which is just using the AI as research 421 00:24:47,920 --> 00:24:50,520 subjects. Not exciting in this case like 422 00:24:50,520 --> 00:24:53,280 they kind of did, but like they basically what they did is that 423 00:24:53,280 --> 00:24:59,000 they aim to replicate 1052 participants. 424 00:24:59,520 --> 00:25:03,080 So they had two hour interviews with each of these participants 425 00:25:03,080 --> 00:25:06,040 in the study. And then the team trained a 426 00:25:06,040 --> 00:25:10,360 generative AI model to mimic their human behavior. 427 00:25:10,800 --> 00:25:14,480 And I think the headline of the study was basically that they 428 00:25:14,480 --> 00:25:20,400 were able to replicate their personality with about 85% 429 00:25:20,400 --> 00:25:24,280 accuracy when it came to tests, surveys and games. 430 00:25:25,160 --> 00:25:29,320 So of course, it's like very confined context that is not 431 00:25:29,320 --> 00:25:32,040 really like representing their personality in like the real 432 00:25:32,040 --> 00:25:34,560 world, but in a very confined setting, they were able to kind 433 00:25:34,560 --> 00:25:38,680 of like basically relatively well mimic the personality of 434 00:25:38,680 --> 00:25:40,520 these people with about two hours of interview. 435 00:25:41,320 --> 00:25:45,480 So the idea was, OK, we replicate these people. 436 00:25:45,480 --> 00:25:49,840 We have a version of them. There's like 8085% reliable. 437 00:25:50,040 --> 00:25:54,640 And then we can then use them as study subjects instead of having 438 00:25:54,640 --> 00:25:57,200 the real people because the real people are kind of expensive. 439 00:25:57,200 --> 00:25:59,760 Or if they're hard to use or they're like maybe unethical to 440 00:25:59,760 --> 00:26:02,800 use for certain studies, but they're AI clones. 441 00:26:02,880 --> 00:26:05,440 We can basically put through another Stanford prison 442 00:26:05,440 --> 00:26:09,480 experiment and you know, I don't think they would even do go 443 00:26:09,480 --> 00:26:11,800 there because Stanford I think they want to keep away from 444 00:26:13,080 --> 00:26:16,240 Stanford person experiment 2 point O, but but technically 445 00:26:16,240 --> 00:26:18,760 they they could in this case. So that was kind of the 446 00:26:18,760 --> 00:26:21,080 headline. And so it was kind of an 447 00:26:21,080 --> 00:26:24,120 interesting study, I think in terms of what is being kind of 448 00:26:24,120 --> 00:26:25,840 explored currently for this stuff. 449 00:26:25,880 --> 00:26:27,200 That's a super interesting study. 450 00:26:27,440 --> 00:26:29,280 It's very similar to the AI alignment project that I'm 451 00:26:29,280 --> 00:26:31,400 working on, which is about aligning to specific 452 00:26:31,400 --> 00:26:36,080 individuals, not to human values in general, which a lot of AI 453 00:26:36,080 --> 00:26:41,200 alignment projects are about general values, but about how 454 00:26:41,200 --> 00:26:43,840 can you be like this specific person. 455 00:26:44,520 --> 00:26:47,960 And in some ways, it's kind of the easy problem of AI 456 00:26:47,960 --> 00:26:52,080 alignment. You can nudge an AI to be more 457 00:26:52,080 --> 00:26:56,720 like someone fairly easily, especially if you have quite a 458 00:26:56,720 --> 00:26:58,920 bit of training data or it's a narrow domain. 459 00:26:59,520 --> 00:27:03,880 If there's only so many options. Does that generalize? 460 00:27:03,880 --> 00:27:06,520 How well does that generalize? I think that's a big question. 461 00:27:07,440 --> 00:27:11,720 Just 'cause I know how you're going to act in a few 462 00:27:11,720 --> 00:27:14,040 psychological studies doesn't necessarily mean I know how 463 00:27:14,040 --> 00:27:17,520 you'll act in the Stanford Prison experiment, for example, 464 00:27:18,680 --> 00:27:22,840 or how you would make triage decisions or any other very 465 00:27:22,840 --> 00:27:25,560 complex topic. So yeah, it, it, it is 466 00:27:25,560 --> 00:27:27,960 interesting. I'm very curious how well it 467 00:27:27,960 --> 00:27:32,560 generalizes. I'm not sure how you know, most 468 00:27:32,560 --> 00:27:35,560 of psychology is about trying to figure out how people do things 469 00:27:35,560 --> 00:27:37,600 and you know why they do what they do. 470 00:27:37,600 --> 00:27:39,400 And we're just not very good at it. 471 00:27:39,800 --> 00:27:43,000 And it'll be interesting to see if AI is able to be better than 472 00:27:43,000 --> 00:27:45,320 humans. Yeah. 473 00:27:45,800 --> 00:27:48,560 And if it will be like, I think from our perspective as 474 00:27:48,560 --> 00:27:52,040 behavioral scientist, you know, the curious case would be 475 00:27:52,520 --> 00:27:57,320 whether they replicate or both better angels, but also like our 476 00:27:57,320 --> 00:28:01,040 vices. I will say that as a personal 477 00:28:01,040 --> 00:28:04,880 experiment, I did actually set up something that I never 478 00:28:04,880 --> 00:28:07,560 actually launched, but I basically set up, I mentioned 479 00:28:07,560 --> 00:28:10,320 before, like an AI voice clone of myself. 480 00:28:10,440 --> 00:28:13,720 And then I set up like a voice agent that you could like I 481 00:28:13,720 --> 00:28:16,600 could send to have calls, like soon calls or stuff like this. 482 00:28:17,080 --> 00:28:20,640 And then because I have for have a weekly pro, I've been doing 483 00:28:20,680 --> 00:28:24,680 mentor calls for the last three years and I have transcript for 484 00:28:24,680 --> 00:28:27,480 all of those calls. I basically summarized those 485 00:28:27,480 --> 00:28:31,080 transcript and used all of this transcript as kind of rag 486 00:28:31,760 --> 00:28:34,640 database or, or training data for, for this AI agent. 487 00:28:35,120 --> 00:28:37,560 And then I gave it some instructions as well to act as 488 00:28:37,560 --> 00:28:42,840 kind of like you're acting as Sam spam, a kind of AI clone of 489 00:28:42,840 --> 00:28:46,280 Samuel as kind of doing mentorship calls. 490 00:28:46,720 --> 00:28:51,080 And it was not really to ever like send it out and, and do it, 491 00:28:51,080 --> 00:28:53,120 but, but it was like, OK, I have a lot of data on this. 492 00:28:53,400 --> 00:28:56,560 Could I create something that would, you know, work and then 493 00:28:56,560 --> 00:29:00,240 see how good it would be? And I, I tested it quite 494 00:29:00,240 --> 00:29:03,400 extensively and it was like, I was impressed with the advice at 495 00:29:03,400 --> 00:29:05,280 some point, like, OK, that's a little better advice than I 496 00:29:05,280 --> 00:29:09,080 would give really good advice. So, yeah, if you're interested, 497 00:29:09,080 --> 00:29:11,400 anyone you can e-mail me and I can give you a link to try it 498 00:29:11,400 --> 00:29:13,200 out. But it was definitely something 499 00:29:13,200 --> 00:29:15,280 that was kind of wild to see what it can do. 500 00:29:16,400 --> 00:29:19,800 But I wouldn't like ethically, like I could never still 501 00:29:19,800 --> 00:29:23,240 ethically do that. I have teased around some 502 00:29:23,240 --> 00:29:25,240 version of like what would be the ethical version of this, but 503 00:29:25,240 --> 00:29:30,520 I just feels very odd to to in any way kind of do some version 504 00:29:30,520 --> 00:29:34,080 of that where you kind of create an AI version of yourself and 505 00:29:34,080 --> 00:29:37,600 kind of send it somewhere. It feels still very dystopian. 506 00:29:38,840 --> 00:29:41,520 So why exactly do you feel it's unethical? 507 00:29:41,520 --> 00:29:44,600 Is it because of the hallucinations or because it 508 00:29:44,600 --> 00:29:46,120 might reference something you said early? 509 00:29:46,120 --> 00:29:47,680 But now you've changed your thinking? 510 00:29:47,960 --> 00:29:49,400 What? What exactly is the concern? 511 00:29:50,440 --> 00:29:54,520 I think the one version of this that I felt more comfortable 512 00:29:54,520 --> 00:29:58,560 with is where I've used it to create a video that basically 513 00:29:58,560 --> 00:30:02,280 has a video avatar on me that looks like a Pixar version of 514 00:30:02,280 --> 00:30:04,840 me. Because then it's very clear 515 00:30:04,840 --> 00:30:06,800 they're like, OK, it sounds like me. 516 00:30:07,160 --> 00:30:09,800 It kind of looks at me, but always like an AI avatar because 517 00:30:09,800 --> 00:30:12,600 it's, it's animated. So anyone who looks at this will 518 00:30:12,600 --> 00:30:16,240 never get confused with me trying to intend that it's 519 00:30:16,240 --> 00:30:19,960 actually me. But however you set things up, 520 00:30:21,040 --> 00:30:24,200 you know, you never know to what degree someone is going to 521 00:30:24,200 --> 00:30:25,920 understand that they're talking with an AI version of you. 522 00:30:26,080 --> 00:30:32,240 Or like that I feel like will be the scary part if like I somehow 523 00:30:32,240 --> 00:30:34,720 set something up and people don't really understand that 524 00:30:34,720 --> 00:30:37,720 you're talking to an AI clone or someone who's not a real person 525 00:30:38,160 --> 00:30:41,240 and they have a really meaningful conversation like my 526 00:30:41,240 --> 00:30:42,440 nightmare. And someone would be, someone 527 00:30:42,440 --> 00:30:45,560 would be like emailing me being like, Oh my God, what you said 528 00:30:45,560 --> 00:30:47,440 the other week was like, changed my life. 529 00:30:47,960 --> 00:30:50,440 Thank you so much. And I'm like, we have never 530 00:30:50,440 --> 00:30:52,080 spoken before. Like you must have accidentally 531 00:30:52,080 --> 00:30:56,200 spoken to my AIA clone. Because then it would just feel 532 00:30:56,200 --> 00:30:58,440 like they've been tricked. And they would feel like such a 533 00:30:58,840 --> 00:31:01,160 weird thing because they've had a meaningful moment. 534 00:31:01,160 --> 00:31:04,200 And that is also that kind of weird thing right now where we 535 00:31:04,200 --> 00:31:07,200 can have meaningful moments with AI content. 536 00:31:07,200 --> 00:31:10,160 And that's OK, I think as long as you know that it's an AI 537 00:31:10,160 --> 00:31:11,280 you're speaking to, right? I don't know. 538 00:31:11,280 --> 00:31:12,600 Does that make sense? Yeah. 539 00:31:12,600 --> 00:31:16,400 So if you could prompt it so that it would be like, hi, I'm 540 00:31:16,600 --> 00:31:21,440 Sam AI, I'm sure there's a good nickname you could give it. 541 00:31:21,920 --> 00:31:26,960 Sam spam. Yeah, Sam spam something that's 542 00:31:26,960 --> 00:31:29,080 under our mats. Our recent guest of the podcast 543 00:31:29,080 --> 00:31:32,120 said where, you know, the difference between science and 544 00:31:32,120 --> 00:31:35,720 science fiction is timing. And it was like, OK, timing has 545 00:31:35,720 --> 00:31:38,720 shifted here. Like, things are becoming quite 546 00:31:38,720 --> 00:31:42,840 strange. Then we'll skip talking about 547 00:31:42,920 --> 00:31:48,360 synthetic content and AI slop. I think everyone can relate to 548 00:31:48,360 --> 00:31:52,000 this. It's just become so much from on 549 00:31:52,000 --> 00:31:57,200 Pinterest to X and Reddit to, you know, whatever. 550 00:31:57,200 --> 00:32:00,000 I think, I think the one thing we can say, because I know you 551 00:32:00,120 --> 00:32:04,040 made a note of it is the AI influencers things on Instagram. 552 00:32:04,720 --> 00:32:08,680 Absolutely atrocious. I have no idea why Meta thinks 553 00:32:08,680 --> 00:32:13,000 this is a good idea. No one wants to see AI avatars 554 00:32:13,000 --> 00:32:15,280 on Instagram. That's not why anyone goes to 555 00:32:15,280 --> 00:32:21,920 Instagram. I like, is it just extreme, like 556 00:32:23,920 --> 00:32:28,440 arrogance or is it just a total misunderstanding of human nature 557 00:32:28,440 --> 00:32:32,240 of their own product? I mean, Facebook has been going 558 00:32:32,480 --> 00:32:35,000 downhill for a while while now. And maybe they're just like, 559 00:32:35,000 --> 00:32:37,280 kind of like desperate to get something. 560 00:32:37,280 --> 00:32:40,200 I mean, it's already I, I, I don't know about other people's 561 00:32:40,200 --> 00:32:46,720 Facebook, but when I get on Facebook, it's about 75% ads or 562 00:32:47,160 --> 00:32:50,560 here's a, you know, a group that you should join, like meme pages 563 00:32:50,560 --> 00:32:52,960 and stuff like that. It's very rare that I see actual 564 00:32:52,960 --> 00:32:57,600 content from my friends anymore. And maybe they're realizing 565 00:32:57,600 --> 00:33:01,200 that's an issue and that they can somehow get around that 566 00:33:01,200 --> 00:33:05,640 issue and that make people perceive that Facebook is 567 00:33:05,640 --> 00:33:10,240 popular again. But Oh my gosh, it's just no one 568 00:33:10,240 --> 00:33:12,760 wants to see what an AI has been doing. 569 00:33:14,320 --> 00:33:17,080 Yeah. And if you missed this, because 570 00:33:17,080 --> 00:33:19,440 this was kind of like they tried to cover up this really quickly, 571 00:33:19,440 --> 00:33:23,760 but the base launched, I think a handful or more AI influencers 572 00:33:23,760 --> 00:33:28,920 that were basically AI agents set up in Instagram with like 573 00:33:28,920 --> 00:33:34,720 really cringy names like dog mom or whatever, like various stuff 574 00:33:34,720 --> 00:33:38,960 like this. And even like, it's a very 575 00:33:38,960 --> 00:33:43,320 specific take, but the even for some reason had one of the AI 576 00:33:43,320 --> 00:33:48,960 avatars had the face of an famous MMA fighter called Israel 577 00:33:49,000 --> 00:33:51,760 Adesanya or Salbunder. Because I, I used to train a 578 00:33:51,760 --> 00:33:54,440 little bit of all the stuff that I happened to know. 579 00:33:54,600 --> 00:33:57,400 And I was like, they have even failed this much on the launch 580 00:33:57,400 --> 00:34:00,920 that they just taking a real person's face for this, you 581 00:34:00,920 --> 00:34:04,280 know, content and whatever they thought was going to happen, 582 00:34:04,280 --> 00:34:07,640 obviously like really backfired. And within 24 hours they 583 00:34:07,640 --> 00:34:11,760 backtracked and they shut it down and and so on. 584 00:34:12,120 --> 00:34:15,400 It reminds me of this version of I think it was Google where they 585 00:34:15,400 --> 00:34:17,920 launched the form of or maybe Twitter where they launched the 586 00:34:17,920 --> 00:34:22,760 form of chatbot that was kind of like basically growing with its 587 00:34:22,760 --> 00:34:25,280 conversations. The idea was that one, it was 588 00:34:25,280 --> 00:34:27,880 going to speak with its users. It's going to learn from how it 589 00:34:27,880 --> 00:34:31,840 speaks and then it's going to become a more of a kind of 590 00:34:31,840 --> 00:34:36,440 growing individual. But like as naive or as tone 591 00:34:36,440 --> 00:34:37,880 deaf as some of these people can be. 592 00:34:37,880 --> 00:34:41,280 Similarly, in this big tech organizations, they couldn't 593 00:34:41,280 --> 00:34:45,520 imagine that like immediately this became like a racist 594 00:34:46,040 --> 00:34:51,040 bigotry spewing like terrible person this chatbot and that 595 00:34:51,040 --> 00:34:54,800 close down within like 12 hours. Yeah, immediately the trolls got 596 00:34:54,800 --> 00:34:58,680 a handle of it because, yeah, I mean, it's the perfect 597 00:34:58,680 --> 00:35:03,480 opportunity for a troll that they can just completely alter 598 00:35:03,600 --> 00:35:06,920 the way this agent is interacting with people. 599 00:35:07,360 --> 00:35:09,680 It was, it was just too perfect an opportunity for the trolls to 600 00:35:09,680 --> 00:35:12,520 leave alone. And I think that's probably a 601 00:35:12,520 --> 00:35:15,080 trend we'll see in the future as well. 602 00:35:16,000 --> 00:35:18,880 Yeah, we have now I think set the scene a little bit. 603 00:35:18,880 --> 00:35:20,800 We've talked about some of the things we noticed in the past 604 00:35:20,800 --> 00:35:24,600 year already, shared some opinions about these things, and 605 00:35:24,600 --> 00:35:26,320 we're going to get into predictions. 606 00:35:26,680 --> 00:35:33,040 But unbeknownst to you, Jared, I prepared a game for us because 607 00:35:33,080 --> 00:35:36,120 we've now talked a lot about like generative AI and all this 608 00:35:36,120 --> 00:35:41,480 stuff. And you're American and I have a 609 00:35:41,480 --> 00:35:47,600 game, which is can you guess the song, whether it's an AI song or 610 00:35:47,600 --> 00:35:50,040 a Eurovision Song? OK. 611 00:35:50,720 --> 00:35:54,320 Do you know what your vision is? I do, but I've never watched it. 612 00:35:54,880 --> 00:35:56,680 OK. What is your what? 613 00:35:56,680 --> 00:35:58,400 What? Tell me, what is your vision to 614 00:35:58,400 --> 00:35:59,280 you? What does that mean for you? 615 00:36:00,720 --> 00:36:04,080 From my understanding, Eurovision is the various 616 00:36:04,080 --> 00:36:07,360 countries in Europe have a artist that gets kind of 617 00:36:07,360 --> 00:36:11,560 nominated for that year and they have a song and they sing this 618 00:36:11,560 --> 00:36:16,720 song at the Eurovision contest, and the Best Song that year wins 619 00:36:16,880 --> 00:36:18,640 the Eurovision. So it's kind of like American 620 00:36:18,640 --> 00:36:21,720 Idol, but for all of the countries where every country 621 00:36:21,720 --> 00:36:25,000 has a Rep. And my understanding is also 622 00:36:25,000 --> 00:36:29,920 that the music videos can sometimes be a little over the 623 00:36:29,920 --> 00:36:32,920 top. Yeah, and we're like the dance 624 00:36:32,920 --> 00:36:35,280 performances. The whole idea is that it's not 625 00:36:35,280 --> 00:36:37,840 really the Best Song per SE, but like it's a mix of what's most 626 00:36:37,840 --> 00:36:40,880 entertaining or kind of most out there creative. 627 00:36:40,880 --> 00:36:44,960 Like if you get a fair mix of basically, you name it, like 628 00:36:46,440 --> 00:36:48,920 any, any given your vicious song that does have basically the 629 00:36:49,360 --> 00:36:51,680 moments of like, what the heck am I watching kind of thing. 630 00:36:52,840 --> 00:36:54,600 But it's been around for a long time. 631 00:36:55,920 --> 00:36:59,880 It's well known that Sweden has won the most Eurovision of all 632 00:36:59,880 --> 00:37:03,440 countries. So we are the the best country 633 00:37:03,720 --> 00:37:06,640 at Eurovision. And so I, I thought I will kind 634 00:37:06,640 --> 00:37:08,040 of play this game with you basically. 635 00:37:08,080 --> 00:37:13,840 And so I will basically play 2 tracks, about 30 seconds for 636 00:37:13,840 --> 00:37:17,720 each track. And you will, because I know you 637 00:37:17,720 --> 00:37:20,680 love predictions and prediction markets and all of this stuff. 638 00:37:21,200 --> 00:37:24,480 So after every song, I'll hit pause and you get a chance to 639 00:37:24,560 --> 00:37:29,040 basically indicate from zero to 100 whether you think it's 640 00:37:29,040 --> 00:37:33,360 human. So 100 is human zero that you 641 00:37:33,360 --> 00:37:35,480 you're very confident that is it's AI basically. 642 00:37:36,120 --> 00:37:40,760 And yeah, then we'll see which, which song has the highest score 643 00:37:40,760 --> 00:37:43,520 and which one you pick is the one that you think is your 644 00:37:43,520 --> 00:37:45,920 revision. Any questions? 645 00:37:47,400 --> 00:37:49,400 Sounds like a musical touring test. 646 00:37:49,400 --> 00:37:53,160 I am very excited. I am not particularly a musical 647 00:37:53,160 --> 00:37:57,880 person. I am very ignorant of the world 648 00:37:57,880 --> 00:38:00,280 of music, so we'll see how I do. I'm. 649 00:38:00,280 --> 00:38:03,000 I'm probably the wrong person for this, but I'll give it my 650 00:38:03,000 --> 00:38:04,600 best. Well, to be fair, like you're 651 00:38:04,600 --> 00:38:08,520 not being introduced to the kind of high fine dining of music 652 00:38:08,520 --> 00:38:10,840 here. Like this is The Dirty, how do 653 00:38:10,840 --> 00:38:13,760 you say like fast food of of music world. 654 00:38:13,760 --> 00:38:17,720 So your palette should be fine for this and as a listener, you 655 00:38:17,720 --> 00:38:20,320 can do the same. So feel free to join and and you 656 00:38:20,320 --> 00:38:22,560 make your predictions along with Jared. 657 00:38:23,760 --> 00:38:32,080 Here we go, song #1. You say that you're real. 658 00:38:33,240 --> 00:38:41,720 Could this be Gas Lion? I can't deny 1 feel how I am 659 00:38:42,640 --> 00:38:50,280 through horrifies. Each whispered clue leaves me 660 00:38:50,280 --> 00:38:54,800 stuck in between. I hate how I love you still 661 00:38:54,800 --> 00:38:58,280 Austin. The screen is the moment 662 00:39:00,200 --> 00:39:06,160 artificial real life. I couldn't see what I can't feel 663 00:39:06,440 --> 00:39:21,170 and I hate how I love you. OK, so that's the time to make 664 00:39:21,170 --> 00:39:29,410 your prediction. 0% if it's AI, 100% if it's human, or somewhere 665 00:39:29,410 --> 00:39:34,530 in between if you're unsure. OK, ready for the next one? 666 00:39:36,290 --> 00:39:40,370 Let's do it. Here we go. 667 00:39:40,370 --> 00:39:43,960 Don't you feel the answer? Judy, we go. 668 00:39:44,920 --> 00:39:50,320 Can you feel it? Go, Judy, we go. 669 00:39:51,560 --> 00:39:56,040 Let me show you our new world. Can't you see it? 670 00:39:56,040 --> 00:39:59,760 Can't you see it? This is the moment. 671 00:40:00,200 --> 00:40:03,680 It's so real. Eyes are burning. 672 00:40:03,720 --> 00:40:06,320 Trees light. Come together. 673 00:40:06,920 --> 00:40:09,960 Let's unite, Judy. We rise. 674 00:40:10,560 --> 00:40:21,120 We go. So let's just get to the scene. 675 00:40:21,720 --> 00:40:25,240 One song was competing in the finals of last year's Eurovision 676 00:40:25,600 --> 00:40:31,800 and the other song was AI generated by myself and using no 677 00:40:31,800 --> 00:40:35,200 sample. Use the prompt and song lyrics 678 00:40:35,200 --> 00:40:38,800 to So no AI, which was the platform I used. 679 00:40:39,360 --> 00:40:46,640 What is your prediction? So I guessed with 85% confidence 680 00:40:46,640 --> 00:40:51,600 that it was human the first song, and I was 45% confident 681 00:40:51,600 --> 00:40:56,000 that the second one was human. OK, so if you had to choose, you 682 00:40:56,000 --> 00:40:58,360 would. I would choose the first one. 683 00:40:58,680 --> 00:41:01,000 Yeah. So now that you're telling me, I 684 00:41:01,120 --> 00:41:02,760 there's just, you know, it's 5050. 685 00:41:02,760 --> 00:41:05,080 I can change my estimates to be more exact. 686 00:41:05,400 --> 00:41:08,000 I'm going to go. But I'm 80% confident that the 687 00:41:08,000 --> 00:41:10,520 first one is human and 20% confident that the second one is 688 00:41:10,520 --> 00:41:13,600 human. OK, well, I'm pleased to say 689 00:41:13,600 --> 00:41:16,200 that you know your Eurovision, Jared. 690 00:41:16,200 --> 00:41:21,000 Well done, well done. And I also have to say now, 691 00:41:21,000 --> 00:41:23,960 listener, I'm sorry, actually, neither of the songs you heard 692 00:41:23,960 --> 00:41:28,440 were from your vision. Because we don't have that 693 00:41:28,440 --> 00:41:30,680 Eurovision money, we can't license the Eurovision songs. 694 00:41:31,080 --> 00:41:36,160 So the listeners heard two AI songs, but the one you heard, 695 00:41:36,160 --> 00:41:40,480 Jared, the first one was Austria's song from last year, 696 00:41:40,880 --> 00:41:48,000 Colleen, she's sang We Will Rave was the the anthem that she had. 697 00:41:48,440 --> 00:41:50,280 And I don't know, it didn't do too well. 698 00:41:50,280 --> 00:41:52,520 Honestly. I think it was like some middle 699 00:41:53,480 --> 00:41:57,000 performing. So it wasn't the best that your 700 00:41:57,000 --> 00:41:59,120 vision had author, but it was in the final. 701 00:41:59,120 --> 00:42:01,800 Like it was voted as one of the top songs obviously that year. 702 00:42:01,800 --> 00:42:05,040 So so yeah, you got it right, Jared. 703 00:42:05,040 --> 00:42:08,800 Good job. I am very proud of myself. 704 00:42:08,840 --> 00:42:12,520 I'm going to put this on my LinkedIn that I am still able to 705 00:42:12,520 --> 00:42:17,200 pass the Turing tests for music. Yeah, yeah. 706 00:42:18,840 --> 00:42:21,520 And I look forward to at some point hosting you in Sweden and 707 00:42:21,520 --> 00:42:23,320 watching Eurovision together. It's a blast. 708 00:42:23,320 --> 00:42:26,480 It's a fun experience. It's Cotton Eye, Joe. 709 00:42:26,720 --> 00:42:29,320 Was that Eurovision? It was a Swedish song. 710 00:42:29,360 --> 00:42:31,680 I don't think they were competing Eurovision. 711 00:42:31,920 --> 00:42:34,080 I'm not sure. But yeah, this was a Swedish 712 00:42:34,080 --> 00:42:38,160 band called Rednecks and they they made a few hits in America 713 00:42:38,960 --> 00:42:42,880 and and yeah, don't listen to it. 714 00:42:43,160 --> 00:42:46,680 I would advise anyone against, but but your vision is coming up 715 00:42:46,760 --> 00:42:49,160 in a few months. So please, please listen and 716 00:42:49,160 --> 00:42:50,600 please watch next year's Your Vision. 717 00:42:51,360 --> 00:42:54,480 OK, Jared, do you feel warmed up and ready to make some 718 00:42:54,480 --> 00:42:56,400 predictions now? Let's do it. 719 00:42:57,560 --> 00:43:03,000 So we're again early in 2025 looking into what seems to be a 720 00:43:03,000 --> 00:43:07,400 kind of a horrid predicted year. It feels very foggy where like 721 00:43:07,400 --> 00:43:09,600 there's a lot of lightnings at the horizon, like a lot of 722 00:43:09,600 --> 00:43:11,480 things are going on, but you can't really tell exactly what's 723 00:43:11,480 --> 00:43:15,520 going to happen because there's so much going on and, and we're 724 00:43:15,520 --> 00:43:17,560 going to do our best. And so I picked out a few 725 00:43:17,560 --> 00:43:20,440 subjects that I feel like these are interesting subjects to make 726 00:43:20,440 --> 00:43:23,120 some thoughts and predictions and reflections on. 727 00:43:23,560 --> 00:43:25,320 And I also really want to hear your thoughts. 728 00:43:26,520 --> 00:43:34,800 The first one, will it become a thing to have AI Co workers in 729 00:43:34,800 --> 00:43:38,000 2025? So what I mean by that is that 730 00:43:38,560 --> 00:43:42,520 AI agents as we talked about have started to become more 731 00:43:42,680 --> 00:43:46,800 widely reduced. But the big limitation has been 732 00:43:47,120 --> 00:43:51,040 infrastructure that the integration of these agents has 733 00:43:51,040 --> 00:43:55,240 been hard because there wasn't really any kind of AI agent 734 00:43:55,240 --> 00:43:59,200 friendly platforms. There wasn't like you had to 735 00:43:59,200 --> 00:44:03,400 like kind of sticky tape it in some ways to make it work, but 736 00:44:03,560 --> 00:44:06,120 it was also a lot of data security issues for a lot of big 737 00:44:06,120 --> 00:44:08,360 organizations. And so it was quite a lot of 738 00:44:08,600 --> 00:44:12,160 hesitation so far to actually integrate them in teams or to 739 00:44:12,160 --> 00:44:15,800 have like, let's say research assistant on Slack. 740 00:44:15,800 --> 00:44:17,840 That's like AI research assistant that anyone in the 741 00:44:17,840 --> 00:44:20,520 team could like ask a research question to, for example. 742 00:44:21,400 --> 00:44:28,000 But many predict that this will be the year for AI coworkers and 743 00:44:28,480 --> 00:44:31,920 Salesforce announced two things. They announced that they will 744 00:44:32,280 --> 00:44:37,840 stop hiring engineers and they will start hiring sales reps to 745 00:44:37,840 --> 00:44:44,960 sell AI agents to organizations. Meta has planned to reduce mid 746 00:44:44,960 --> 00:44:48,480 level engineers. Zuckerberg was talking about 747 00:44:48,480 --> 00:44:51,840 this and Rogan recently saw some clip around. 748 00:44:51,840 --> 00:44:55,000 He was kind of bragging about like, yeah, we'll be able to cut 749 00:44:55,000 --> 00:44:57,880 out whale of mill level engineering and use a A agents 750 00:44:57,880 --> 00:45:02,000 instead. And so there's some signals 751 00:45:02,000 --> 00:45:05,840 around that this is likely to be a trend this year, certainly 752 00:45:05,840 --> 00:45:08,440 hype around it. But what are things, Jared? 753 00:45:08,560 --> 00:45:13,000 Do you think there will be AI Co workers popping up in 2025? 754 00:45:14,760 --> 00:45:17,800 The resolution criteria for this is difficult because what what 755 00:45:17,800 --> 00:45:21,440 counts as a Co worker? Certainly it's going to be the 756 00:45:21,440 --> 00:45:25,760 case that companies are going to be increasingly using some kind 757 00:45:25,760 --> 00:45:28,480 of chat bot. They're increasingly going to be 758 00:45:28,480 --> 00:45:32,840 using agents that are programming that are doing lots 759 00:45:32,840 --> 00:45:36,800 of different tasks. Are people going to think, oh, 760 00:45:36,880 --> 00:45:40,200 here's my AI Co worker, or are they going to think, here's this 761 00:45:40,200 --> 00:45:44,960 tool that I'm using? It's not clear to me like how 762 00:45:45,000 --> 00:45:50,240 how you make that distinction. So I, I don't think we're going 763 00:45:50,240 --> 00:45:53,280 to get to the point where people are going to be thinking where 764 00:45:53,800 --> 00:45:57,960 an agent is autonomous enough that it feels like a Co worker. 765 00:45:58,640 --> 00:46:01,160 Rather it's only going to be autonomous enough to make it 766 00:46:01,160 --> 00:46:04,520 feel like it's a tool. So I'm, I'm I'm going to predict 767 00:46:04,600 --> 00:46:08,200 no because people aren't going to think of them as Co workers. 768 00:46:09,840 --> 00:46:13,720 What if it was? So I have done with some tools, 769 00:46:13,720 --> 00:46:17,480 for example, that you're on a call like this and there's a 770 00:46:17,480 --> 00:46:20,800 third person on the call and you can give them access to certain 771 00:46:20,800 --> 00:46:22,320 things. You can have access to the web. 772 00:46:22,320 --> 00:46:24,800 You can give access to some like actually some internal 773 00:46:24,800 --> 00:46:26,880 documentation and keep it like local. 774 00:46:27,400 --> 00:46:33,600 And maybe some like tasks like maybe be able to book calendar 775 00:46:33,600 --> 00:46:35,040 appointments or or various things. 776 00:46:35,520 --> 00:46:38,680 And so we could have in the same way as people are starting to 777 00:46:38,680 --> 00:46:42,720 have like no takers that you can now very likely this year at 778 00:46:42,720 --> 00:46:46,760 some point have your no taker actually be able to execute on 779 00:46:46,760 --> 00:46:50,440 task for you on call. And so you could say like, Hey, 780 00:46:50,440 --> 00:46:54,120 Alexa, you know, add something to my shopping list is something 781 00:46:54,120 --> 00:46:56,520 we we've been saying for a while, but now you can more 782 00:46:56,520 --> 00:47:03,280 like, hey, Fred, book a calendar appointment for next Thursday or 783 00:47:03,560 --> 00:47:05,680 can you look up this thing or like, what did we say last 784 00:47:05,680 --> 00:47:08,040 meeting? Does that feel like a Co worker? 785 00:47:08,040 --> 00:47:10,080 Like does that take the box of a Co worker? 786 00:47:10,840 --> 00:47:12,960 It does still doesn't feel like a Co worker to me. 787 00:47:12,960 --> 00:47:15,760 I would still treat that like a tool if, if that's how you're 788 00:47:15,760 --> 00:47:18,960 defining Co worker, I would say 100% we're we're going to get 789 00:47:18,960 --> 00:47:23,640 that, but that people will feel like it's a Co worker rather 790 00:47:23,640 --> 00:47:28,400 than just telling a, you know, an interface to do something and 791 00:47:28,400 --> 00:47:30,280 that a voice is just now the interface. 792 00:47:32,600 --> 00:47:35,040 I, I don't think people will think of that as a Co worker. 793 00:47:35,040 --> 00:47:36,480 It's that's just another interface. 794 00:47:37,360 --> 00:47:42,240 OK, and what if let's seems like there was a person that could 795 00:47:42,240 --> 00:47:47,160 perform all tasks of let's say a research assistant or a social 796 00:47:47,160 --> 00:47:52,320 media manager, like this non person, this agent can perform 797 00:47:52,320 --> 00:47:56,240 all the tasks of this specific profession. 798 00:47:56,920 --> 00:48:00,160 But it's still you know that it's an AI and you communicate 799 00:48:00,160 --> 00:48:03,040 mainly with shat. But maybe it's given a name and 800 00:48:03,040 --> 00:48:08,080 maybe has some form of photo, like some form of cartoon photo 801 00:48:08,080 --> 00:48:12,120 or something. Let's call again Fred. 802 00:48:13,160 --> 00:48:16,920 Would you see Fred as some form of like colleague that helps you 803 00:48:16,920 --> 00:48:18,680 with your work or would you still see this as a tool? 804 00:48:19,840 --> 00:48:23,320 I I would still see as a tool but I could also imagine it 805 00:48:23,320 --> 00:48:26,720 getting to the point and adding enough personalization elements. 806 00:48:27,040 --> 00:48:30,640 The people start saying it's their Co worker so I think we 807 00:48:30,640 --> 00:48:34,280 could get to that point. I am skeptical that we can 808 00:48:34,280 --> 00:48:40,240 outsource an entire role like that, at least not yet. 809 00:48:40,480 --> 00:48:44,280 But hey, 2025 is that's a long time. 810 00:48:44,920 --> 00:48:47,960 One year is a long time in AI years at this point. 811 00:48:48,920 --> 00:48:51,960 Yeah, I, I can't promise you, Jared, this, this can be 812 00:48:51,960 --> 00:48:54,880 something we can follow up on, but I can promise you that we're 813 00:48:54,880 --> 00:48:57,960 in within our slack space. I will at some point this year 814 00:48:57,960 --> 00:49:02,920 set up some version of this and so that will test and see how 815 00:49:02,920 --> 00:49:06,360 you feel about it. All right, so here should we 816 00:49:06,360 --> 00:49:09,880 make a bet? You know, at the end of 2025, we 817 00:49:09,880 --> 00:49:14,800 should pull our coworkers and say, you know, is this a Co 818 00:49:14,800 --> 00:49:18,080 worker? Is our AI friend a Co worker or 819 00:49:18,160 --> 00:49:20,280 is it a tool? And we'll and we'll see what 820 00:49:20,560 --> 00:49:22,080 Elaine, Rose and Hassan have to say. 821 00:49:22,480 --> 00:49:24,920 Yes, that's a good one. What I I think is the tipping 822 00:49:24,920 --> 00:49:27,520 point, what I'm going to include in this is that I'm going to 823 00:49:27,960 --> 00:49:33,400 make sure that it shares memes and and and gifts as well. 824 00:49:33,400 --> 00:49:36,440 So when it communicates it, it sometimes randomly communicates 825 00:49:36,440 --> 00:49:38,680 with gifts and so on. I think that's going to be the 826 00:49:38,680 --> 00:49:41,120 tipping point in my favor. That's going to kind of convince 827 00:49:41,120 --> 00:49:44,600 them that it's a Co worker. We'll see, because if it's if it 828 00:49:44,600 --> 00:49:48,680 feels like slop, right? I mean, if it feels like slop 829 00:49:48,680 --> 00:49:52,080 and it's just like a random GIF or if it's like a meaningful 830 00:49:52,080 --> 00:49:55,680 moment, you know, I'm like, you know, hey guys, I'm announcing 831 00:49:55,680 --> 00:49:57,400 my baby. I just look, here's a cute 832 00:49:57,400 --> 00:50:00,600 picture of my baby. And then you know your AI bot 833 00:50:00,600 --> 00:50:02,440 shares. AI don't know. 834 00:50:03,400 --> 00:50:05,680 Sitting in a fine or meme or something, yeah. 835 00:50:05,720 --> 00:50:12,760 Yeah, like this is fine meme. Yeah, we'll see, We'll see. 836 00:50:12,760 --> 00:50:15,520 We'll see what me it shares that that will be the race to see 837 00:50:15,680 --> 00:50:18,480 what's going to come first your baby or our AI. 838 00:50:20,600 --> 00:50:21,800 OK, but that's interesting though. 839 00:50:21,840 --> 00:50:26,680 I think, I think, I think it's a matter of more how it would look 840 00:50:26,680 --> 00:50:29,280 like more than you know, to what degree there will be some form 841 00:50:29,280 --> 00:50:32,160 of integration, but I think the integration will still be 842 00:50:32,360 --> 00:50:34,960 there's a lot of limitations that we still need to overcome. 843 00:50:35,320 --> 00:50:38,560 And I think especially at scale, larger recessions where like 844 00:50:38,560 --> 00:50:43,040 Nuance were relatively small, you know, consultancy and so on, 845 00:50:43,800 --> 00:50:46,920 we don't have to worry as much about like these various 846 00:50:46,920 --> 00:50:50,480 complicated setups that you would have to pass if you were 847 00:50:50,480 --> 00:50:53,680 like a multi continental. Like we're actually we're 848 00:50:53,880 --> 00:50:58,160 spanning multi continents, but we're quite, you know, leanly 849 00:50:58,160 --> 00:50:59,400 compared to some big organizations. 850 00:50:59,720 --> 00:51:02,240 So I do think there's going to be a lot of stuff that's going 851 00:51:02,240 --> 00:51:05,000 to prevent this from happening at scale still, but we're going 852 00:51:05,000 --> 00:51:07,760 to see some use cases that's going to probably be quite cool 853 00:51:08,160 --> 00:51:09,240 this year. Absolutely. 854 00:51:10,480 --> 00:51:12,680 OK, Then we're getting into science. 855 00:51:13,440 --> 00:51:15,880 We love science, we're behavioral scientists, both of 856 00:51:15,880 --> 00:51:19,240 us. And the question is will AI 857 00:51:19,680 --> 00:51:26,120 reinvent science in 2025? So basically some seeds has been 858 00:51:26,120 --> 00:51:31,480 planted that could potentially lead for AI to I have in my 859 00:51:31,480 --> 00:51:34,440 notes, quote UN quote, supercharged the scientific 860 00:51:34,440 --> 00:51:38,560 process, accelerating literature reviews, data analysis, and even 861 00:51:38,720 --> 00:51:40,400 hypothesis generation and testing. 862 00:51:40,960 --> 00:51:46,000 Will this spark a new age of scientific breakthroughs? 863 00:51:48,240 --> 00:51:50,880 Well, I would say 100% it's already happening. 864 00:51:51,400 --> 00:51:53,880 Scientists are already using it for lit reviews. 865 00:51:54,480 --> 00:51:57,920 I mentioned we, we were talking about Notebook LM earlier, which 866 00:51:57,920 --> 00:52:01,080 is just a really great resource, especially if you just need 867 00:52:01,080 --> 00:52:05,200 something quick. At the end of the day, when I'm 868 00:52:05,200 --> 00:52:07,960 working on a project, I still need to understand the material. 869 00:52:08,480 --> 00:52:12,440 And so I'm still reading scientific articles regularly 870 00:52:12,680 --> 00:52:14,800 and I can't outsource that completely. 871 00:52:15,520 --> 00:52:18,040 But there are occasions where it's just like, hey, I just 872 00:52:18,040 --> 00:52:22,000 need, you know, a lit review on this topic on trust, for 873 00:52:22,000 --> 00:52:26,600 example, on trust of AI agents. And I, I have all these papers 874 00:52:26,600 --> 00:52:30,160 in my Zotero in my database where I keep all of my academic 875 00:52:30,160 --> 00:52:32,720 articles. And I just drop all of those 876 00:52:32,840 --> 00:52:36,840 into notebook LM and say summarize this because I'm not 877 00:52:36,840 --> 00:52:40,440 going to reread all of those articles and I haven't even read 878 00:52:40,440 --> 00:52:42,560 all of them. So, you know, it's just going to 879 00:52:42,560 --> 00:52:44,080 be a lot of work. If I wanted to try and 880 00:52:44,080 --> 00:52:45,560 understand everything that's going on. 881 00:52:45,960 --> 00:52:48,160 And often times what scientists would do would just read the 882 00:52:48,160 --> 00:52:52,840 abstract anyways. So having an, an LMLLM summarize 883 00:52:52,840 --> 00:52:56,040 that is really useful. So we're already at that point 884 00:52:56,040 --> 00:52:58,400 with lit review. We're already at the point where 885 00:52:58,720 --> 00:53:03,240 AI is doing some data analysis. I've done it myself where I 886 00:53:03,240 --> 00:53:05,800 couldn't figure out the R code for something and I just said, 887 00:53:05,800 --> 00:53:10,120 hey, AI do the R code for me. And I figured it out, made some 888 00:53:10,120 --> 00:53:14,560 really pretty graphs for me. And and then hypothesis 889 00:53:14,560 --> 00:53:17,320 generation. I think it's also already there. 890 00:53:18,360 --> 00:53:21,760 I, this is probably my most frequent use case because I'm 891 00:53:21,760 --> 00:53:25,840 such a nerd that most of what I do with ChatGPT is just like, 892 00:53:26,120 --> 00:53:28,680 hey, here's this, you know, hypothesis I have, or here's 893 00:53:28,680 --> 00:53:32,480 this, you know, theory and it, it helps me to think through it. 894 00:53:32,960 --> 00:53:36,200 And so it's already there. Is that going to lead to a 895 00:53:36,240 --> 00:53:39,480 revolution? It's hard to see that. 896 00:53:39,800 --> 00:53:43,320 I think there are various things like I, I think we're already on 897 00:53:43,320 --> 00:53:45,600 the verge of the revolution in behavioral science. 898 00:53:45,600 --> 00:53:49,880 I think people are thinking about behavioral science very 899 00:53:49,880 --> 00:53:51,480 differently than they were five years ago. 900 00:53:51,880 --> 00:53:54,680 And I think at some point there's going to be a switch. 901 00:53:54,680 --> 00:53:58,920 And I think things could change really rapidly in behavioral 902 00:53:58,920 --> 00:54:02,600 science. AI might play a part of it, but 903 00:54:02,600 --> 00:54:04,960 I don't think anyone's going to point to AI as being the thing 904 00:54:04,960 --> 00:54:07,680 that caused that change. And I think that'll be the case 905 00:54:07,680 --> 00:54:09,640 in most of the major changes in science. 906 00:54:10,000 --> 00:54:12,760 That'll be like, well, AI contributed, but it's not going 907 00:54:12,760 --> 00:54:15,880 to be the thing that was like, well, AI is the new Copernicus, 908 00:54:16,640 --> 00:54:18,160 it's the Newton, it's the Einstein. 909 00:54:18,160 --> 00:54:19,840 It's the thing causing the revolution. 910 00:54:21,280 --> 00:54:22,720 That's a really interesting take. 911 00:54:22,720 --> 00:54:26,200 Yeah. And yeah, I agree with 912 00:54:26,200 --> 00:54:27,680 everything I think you said there. 913 00:54:27,800 --> 00:54:31,800 And to add some further signals to what you mentioned, there's 914 00:54:31,800 --> 00:54:36,520 been a research study that came out last year around basically 915 00:54:36,520 --> 00:54:41,200 predicting who does the best job of predicting neuroscience 916 00:54:41,200 --> 00:54:45,960 results in AI or neuroscientists and the AI beat the expert near 917 00:54:45,960 --> 00:54:48,960 scientists in terms of predicting neuroscience finding. 918 00:54:49,760 --> 00:54:57,200 Then we saw AAI generated 288 complete academic finance papers 919 00:54:57,440 --> 00:55:00,360 predicting stock returns, a complete with possible 920 00:55:00,360 --> 00:55:02,000 theoretical frameworks and citations. 921 00:55:03,280 --> 00:55:05,320 I don't know if all of those have been like accepted and 922 00:55:05,320 --> 00:55:10,080 published at any reputable journal, but there is a lot of 923 00:55:10,080 --> 00:55:13,320 like I can go on with a lot of these examples of various 924 00:55:13,320 --> 00:55:17,480 studies that have backed up the idea that not only is being 925 00:55:17,480 --> 00:55:20,920 used, but it's actually able to to perform at certain things 926 00:55:20,920 --> 00:55:23,160 relatively well. And then you referenced kind of 927 00:55:23,480 --> 00:55:26,720 putting your stuff on to notebook LM. 928 00:55:26,960 --> 00:55:32,200 It's also open source tools like Storm from Stanford, where 929 00:55:32,200 --> 00:55:35,320 they're kind of trying to automate some version of 930 00:55:36,000 --> 00:55:38,880 literature review. I think deep research from 931 00:55:38,880 --> 00:55:41,480 Gemini and kind of new mode that they've introduced in their 932 00:55:42,240 --> 00:55:45,720 service is even better. You have also like previously, I 933 00:55:45,720 --> 00:55:48,080 think somewhere in between you have perplexity that has a 934 00:55:48,080 --> 00:55:49,280 decent job for some of these things. 935 00:55:49,280 --> 00:55:52,440 I mean, there's so many different use cases now where 936 00:55:52,520 --> 00:55:58,240 before you had to maybe if you wanted to test AI with science, 937 00:55:58,240 --> 00:56:00,800 you have to ask your GP something and be disappointed 938 00:56:00,800 --> 00:56:03,480 with some hallucination and some fake citation. 939 00:56:03,800 --> 00:56:06,520 But now there's a lot of these tools that are like specifically 940 00:56:06,520 --> 00:56:11,280 being done for for science and actually breaking news open AI 941 00:56:11,280 --> 00:56:17,440 used released a basically AAI model for longevity science. 942 00:56:18,040 --> 00:56:21,800 So they've partnered with an organization specifically 943 00:56:22,400 --> 00:56:26,480 looking at scientific discovery when it comes to build and 944 00:56:26,480 --> 00:56:29,760 manufacture stem cells and longevity research. 945 00:56:30,480 --> 00:56:34,160 And so again, like that's a model that is only kind of 946 00:56:34,840 --> 00:56:37,600 focused on a very specific niche in science. 947 00:56:37,600 --> 00:56:41,480 And we've seen some breakthroughs within biology and 948 00:56:42,280 --> 00:56:46,400 other kind of sciences where there's been like AI model 949 00:56:46,400 --> 00:56:49,320 trainers very specific thing. But again, I think this signals 950 00:56:49,320 --> 00:56:53,080 that that's not only true for more traditional form of AI use 951 00:56:53,080 --> 00:56:57,600 cases, but also these LLM models can enable certain type of 952 00:56:57,600 --> 00:57:00,120 scientific study. And I think that's actually kind 953 00:57:00,120 --> 00:57:03,080 of a significant breakthrough. And that's like, not only can 954 00:57:03,080 --> 00:57:06,880 these LLM models generate content and and perform in 955 00:57:06,880 --> 00:57:09,920 various tests, but they can actually potentially be trained 956 00:57:09,920 --> 00:57:12,320 to be more specifically used for science as well. 957 00:57:13,200 --> 00:57:16,880 There probably are some domains where AI is going to be really 958 00:57:16,880 --> 00:57:19,760 useful and might be the thing that drives the breakthrough. 959 00:57:20,120 --> 00:57:26,840 So if there's a domain where there's 200 genes and scientists 960 00:57:26,840 --> 00:57:30,280 know that it's one of those 200 genes, but they don't know you 961 00:57:30,360 --> 00:57:33,080 know which one it is, that's going to be like the gene that 962 00:57:33,080 --> 00:57:36,240 matters for the certain thing. And AI is going to be able to 963 00:57:36,720 --> 00:57:38,440 analyze that perhaps more rapidly. 964 00:57:38,800 --> 00:57:42,360 Been a scientist. I could imagine, you know, like 965 00:57:42,600 --> 00:57:44,880 we, we already have AI that's, you know, folding proteins. 966 00:57:44,880 --> 00:57:48,560 And you could imagine that there's a breakthrough because 967 00:57:48,560 --> 00:57:52,320 of that work. I, I can imagine it there when 968 00:57:52,320 --> 00:57:55,680 it's a little bit more open-ended, when it's like 969 00:57:55,800 --> 00:57:59,520 fundamentally about the revolution where it's a 970 00:57:59,520 --> 00:58:03,560 restructuring of how we even think about science like, and 971 00:58:03,560 --> 00:58:06,240 the way that Einstein completely re conceptualized how we think 972 00:58:06,240 --> 00:58:11,000 about space and time, that is I think more difficult. 973 00:58:11,480 --> 00:58:16,200 So I think AI might be able to do Nobel Prize worthy work in 974 00:58:16,200 --> 00:58:19,960 that it might find the protein that is the thing that unlocks 975 00:58:20,600 --> 00:58:23,760 some kind of longevity. But it's not going to be able to 976 00:58:23,760 --> 00:58:26,240 do the Einstein type thing where it's able to completely 977 00:58:26,240 --> 00:58:29,960 revolutionize because it's so constrained by its text that 978 00:58:29,960 --> 00:58:33,800 it's trained on that I it'll have difficulty breaking out of 979 00:58:33,800 --> 00:58:37,520 that paradigm. Yeah, I think it's fair. 980 00:58:37,840 --> 00:58:43,040 And I guess what I see as a challenge with what we're 981 00:58:43,040 --> 00:58:47,160 already noticing is that these models are improving relatively 982 00:58:47,160 --> 00:58:51,080 quickly still. So, you know, last year it 983 00:58:51,080 --> 00:58:54,520 started with, let's say, if you take folks on Open AI, which we 984 00:58:54,520 --> 00:58:59,360 talked about, they started with the four O model or at least 4 985 00:58:59,360 --> 00:59:04,160 model and then ended with one O or potentially the three old 986 00:59:04,160 --> 00:59:06,560 model that hasn't actually been officially released. 987 00:59:06,560 --> 00:59:09,800 But like in comparing these models, they actually are quite 988 00:59:09,800 --> 00:59:14,160 different and science doesn't move as quick as some of these 989 00:59:14,160 --> 00:59:17,920 breakthroughs are coming out. And especially like published 990 00:59:17,920 --> 00:59:23,600 signs, like going through, you know, peer reviewed and properly 991 00:59:23,600 --> 00:59:26,880 done signs, not only like preprints on Archivexer, you 992 00:59:26,880 --> 00:59:28,800 know, like not only like the quick stuff that you can just 993 00:59:28,800 --> 00:59:33,800 share quickly, that takes time. And then you're kind of noticing 994 00:59:33,800 --> 00:59:37,600 that the stuff that's coming out in nature today, you saying like 995 00:59:38,560 --> 00:59:42,000 GPT 3 model, for example, And then you're like feeling, well, 996 00:59:42,040 --> 00:59:44,080 what does it like to say today about the current state of 997 00:59:44,080 --> 00:59:46,760 models? And that is kind of a tricky 998 00:59:46,760 --> 00:59:48,960 thing where I feel like there's a lot of talking about slot. 999 00:59:48,960 --> 00:59:52,640 There's a lot of papers that have been published on like that 1000 00:59:52,640 --> 00:59:56,240 feels AI generator almost where they are using some relatively 1001 00:59:56,360 --> 00:59:59,360 simple hypothesis or something that they run and they. 1002 01:00:00,160 --> 01:00:03,240 Send it out to some form of preprints or some form of thing 1003 01:00:03,240 --> 01:00:07,960 where it's like relatively early release without peer reviewing. 1004 01:00:08,360 --> 01:00:10,800 And you're kind of stuck in between those modes where either 1005 01:00:10,800 --> 01:00:12,720 you're reading this kind of research, which is kind of like 1006 01:00:13,240 --> 01:00:16,880 just being sent out there hot from the presses, but actually 1007 01:00:16,880 --> 01:00:20,400 using the latest models, or you're reading the research that 1008 01:00:20,400 --> 01:00:25,840 is often times really like rigorously set up and studied 1009 01:00:25,840 --> 01:00:29,600 from refugee for labs and so on. But was and I think that's 1010 01:00:29,640 --> 01:00:30,880 challenging. We're keeping up with the signs 1011 01:00:30,880 --> 01:00:32,960 today is kind of like that balance. 1012 01:00:32,960 --> 01:00:34,160 I don't know if you noticed that as well. 1013 01:00:35,320 --> 01:00:36,680 There's definitely some papers out. 1014 01:00:36,680 --> 01:00:38,440 There that are. Slop. 1015 01:00:39,360 --> 01:00:42,560 And I mean, that's actually not new. 1016 01:00:42,600 --> 01:00:47,400 There's always been slop papers, but it's the how. 1017 01:00:47,400 --> 01:00:49,840 Blatant. It is at times is really 1018 01:00:49,840 --> 01:00:52,800 shocking. You have this, you know, a, a 1019 01:00:52,800 --> 01:00:57,480 picture of a rat and this paper, and it's just like that. 1020 01:00:57,480 --> 01:00:59,040 Well, that's not the anatomy of a rat. 1021 01:00:59,040 --> 01:01:01,760 Like, like it's just so completely wrong. 1022 01:01:02,160 --> 01:01:04,800 That's shocking that it somehow still got published. 1023 01:01:06,320 --> 01:01:06,960 So. Yeah. 1024 01:01:07,240 --> 01:01:11,640 But Stanford, I'm joking, that was that was actually a non AI 1025 01:01:11,640 --> 01:01:15,840 related controversy, but that was some photoshopping. 1026 01:01:15,840 --> 01:01:19,720 I think it was by someone at Stanford who photoshopping some 1027 01:01:19,720 --> 01:01:22,680 research. But but yeah, no, I agree with 1028 01:01:22,680 --> 01:01:24,720 you. And is there any prediction you 1029 01:01:24,720 --> 01:01:29,320 would want to make around science in 2025 and AI or 1030 01:01:29,680 --> 01:01:33,040 behavioral science? I mean, I'll give it less. 1031 01:01:33,040 --> 01:01:35,600 Than a 1%. Chance that AI wins a Nobel 1032 01:01:35,600 --> 01:01:39,960 Prize, You know that the. AI is fundamentally what? 1033 01:01:40,840 --> 01:01:44,760 Led to the discovery of whatever wins the Nobel Prize. 1034 01:01:44,760 --> 01:01:48,400 Of course Nobel Prize is looking backwards so I maybe I need to 1035 01:01:48,400 --> 01:01:50,480 make that a 5 or 10 year prediction. 1036 01:01:51,040 --> 01:01:54,600 You know, I, I, in the next 5 years, I think it's probably 1037 01:01:54,600 --> 01:01:58,160 less than 1%. But in the next 10, I, I think 1038 01:01:58,360 --> 01:02:02,560 there's, there might be a 10% chance that we have a discovery 1039 01:02:02,560 --> 01:02:06,240 where it's, the AI fundamentally LED that discovery. 1040 01:02:06,520 --> 01:02:08,200 And that's what leads to the Nobel Prize. 1041 01:02:08,960 --> 01:02:11,880 But in terms of the revolutionary work, if we're 1042 01:02:11,880 --> 01:02:17,920 talking about kind of like the paradigms of science, AI has a 1043 01:02:17,920 --> 01:02:21,000 really hard time doing that, of breaking out of its paradigm 1044 01:02:21,280 --> 01:02:24,560 because you provide it context and it sticks to that context. 1045 01:02:24,560 --> 01:02:28,760 It's trying to predict the next word, the next token within the 1046 01:02:28,760 --> 01:02:31,680 context that you have given it. And so it's hard for me to 1047 01:02:31,680 --> 01:02:34,960 imagine that AI is going to be the one that that leads the 1048 01:02:35,080 --> 01:02:38,360 fundamental discovery like that, where it actually revolutionizes 1049 01:02:38,360 --> 01:02:40,440 science. It's so hard to predict. 1050 01:02:40,440 --> 01:02:43,560 So I mean, we, we could have AI that's just so incredible that 1051 01:02:43,560 --> 01:02:45,160 it just blows me away in 10 years. 1052 01:02:46,520 --> 01:02:47,960 But I don't think we'll be there. 1053 01:02:48,520 --> 01:02:51,960 I'm, I don't know if that makes me a pessimist or an optimist, 1054 01:02:52,120 --> 01:02:55,120 but I don't think AI is going to completely revolutionize science 1055 01:02:55,120 --> 01:02:58,480 in the next 10 years. I don't know. 1056 01:02:58,480 --> 01:03:02,120 Where it puts you I think. Somewhere maybe in between if 1057 01:03:02,120 --> 01:03:05,960 you say something like that, I think I think everyone is on the 1058 01:03:05,960 --> 01:03:11,360 same page in terms of like looking further ahead than a few 1059 01:03:11,360 --> 01:03:12,920 years time. It's just really hard to know 1060 01:03:12,920 --> 01:03:17,720 what's going to happen. And I think you you're right in 1061 01:03:17,720 --> 01:03:20,200 many ways. I guess the interesting thing 1062 01:03:20,360 --> 01:03:23,680 that I was just kind of put to you in terms of not sure whether 1063 01:03:23,680 --> 01:03:28,200 it's a still man or a strong man of this, but like do you think 1064 01:03:28,440 --> 01:03:29,800 because. You're talking about AI. 1065 01:03:29,800 --> 01:03:34,320 Basically from almost from zero to 1, creating novel research 1066 01:03:34,320 --> 01:03:36,840 that then becomes maybe potentially in the best case, 1067 01:03:36,840 --> 01:03:41,000 like a Nobel Prize research that is maybe. 1068 01:03:41,000 --> 01:03:44,640 Like. The most strongest version of of 1069 01:03:44,640 --> 01:03:47,800 what AI in science look like, but like maybe turning on on the 1070 01:03:47,800 --> 01:03:50,360 other side of like the weakest, but still involved. 1071 01:03:50,360 --> 01:03:55,080 Like do you think there would be anything that is being 1072 01:03:55,080 --> 01:04:00,600 eventually call it, let's say Nobel Prize winning findings in 1073 01:04:00,880 --> 01:04:04,600 kind of the hard sciences that is being from now on done 1074 01:04:04,720 --> 01:04:08,720 completely without AI involved? Like completely done with? 1075 01:04:08,720 --> 01:04:13,080 Only human. Kind of inputs in terms of not 1076 01:04:13,080 --> 01:04:17,760 in any steps of the way. Are they using AI as as part of 1077 01:04:17,760 --> 01:04:20,160 the discovery? Interesting. 1078 01:04:21,960 --> 01:04:27,920 It seems unlikely AI has become such a fundamental tool. 1079 01:04:29,240 --> 01:04:32,160 There's there's still some holdouts, there's still a lot of 1080 01:04:32,160 --> 01:04:34,760 people in academia who are very anti AI. 1081 01:04:35,920 --> 01:04:38,080 I've definitely noticed that online that there's some very 1082 01:04:38,080 --> 01:04:41,400 strong feelings. And so you and you probably have 1083 01:04:41,400 --> 01:04:45,400 some of the older scientists who are just a little bit slower and 1084 01:04:45,400 --> 01:04:49,840 picking up the new tools. But as the new generation rises, 1085 01:04:50,800 --> 01:04:52,960 you know my generation. Who We didn't grow up. 1086 01:04:52,960 --> 01:04:55,440 With AI, but we're quickly adopting it, and the younger 1087 01:04:55,440 --> 01:04:59,040 generations who are using it to pass their tests in high school, 1088 01:05:01,040 --> 01:05:03,920 you know, they're going to grow up very used to AI. 1089 01:05:04,520 --> 01:05:06,800 And it's hard to imagine that AI is not going to play a 1090 01:05:06,800 --> 01:05:10,000 fundamental part, at least some of the process. 1091 01:05:11,960 --> 01:05:14,280 Yeah. Yeah, I know. 1092 01:05:14,280 --> 01:05:19,480 I, I I. Would also say that and I, I 1093 01:05:19,480 --> 01:05:21,360 would say to the defense. Of the people that are very. 1094 01:05:21,360 --> 01:05:23,520 Skeptical. I think it's great that we're, 1095 01:05:23,520 --> 01:05:26,280 we'll have a little bit of a, a mixed deal because I think there 1096 01:05:26,280 --> 01:05:29,400 needs to be some form of counterbalance like between the, 1097 01:05:29,960 --> 01:05:33,080 you know, the hype and there has to be a little bit of doom and 1098 01:05:33,080 --> 01:05:34,160 gloom. As well or like some form. 1099 01:05:34,160 --> 01:05:36,840 Of hesitancy and skepticism and so on. 1100 01:05:38,280 --> 01:05:40,600 Because. Yeah, there's always. 1101 01:05:40,600 --> 01:05:44,600 Costs of building a a plane while you're flying it right 1102 01:05:44,600 --> 01:05:49,040 like you, you might put on two wings on one side and realizing 1103 01:05:49,040 --> 01:05:51,920 like, oh shit, I need I need one on each side. 1104 01:05:52,720 --> 01:05:54,320 And that's kind of what we're finding a little bit now. 1105 01:05:54,520 --> 01:06:01,560 So this leads us in some ways to the final area of predictions, 1106 01:06:02,520 --> 01:06:08,280 which is really trying to stare blindly into the light of hype, 1107 01:06:08,720 --> 01:06:14,600 which is the hype of AGI and the prediction around kind of the 1108 01:06:14,600 --> 01:06:19,800 AGI debate intensifying. So with open AI kind of 1109 01:06:21,920 --> 01:06:24,400 successfully. Beating what I seem. 1110 01:06:24,400 --> 01:06:30,080 To be at least a maybe not a full AGI perfect benchmark, but 1111 01:06:30,080 --> 01:06:34,360 like a really high threshold benchmark in the arc benchmark 1112 01:06:35,360 --> 01:06:39,040 probably ahead of time from most people's predictions like maybe 1113 01:06:39,040 --> 01:06:41,760 2-3 years before most people would predict that maybe that 1114 01:06:41,760 --> 01:06:44,600 would happen that has kind of like. 1115 01:06:44,600 --> 01:06:48,600 Started a vibe shift and. We're starting to, you know, 1116 01:06:49,320 --> 01:06:54,280 see, I would say some relatively irresponsible communication 1117 01:06:54,280 --> 01:06:58,560 coming out from various labs and various things kind of hinting 1118 01:06:58,560 --> 01:07:02,160 at this stuff like all, you know, soon the Super 1119 01:07:02,160 --> 01:07:03,920 intelligence. Is here or or can you feel? 1120 01:07:03,920 --> 01:07:06,200 The AGI, all of these things are coming out from like 1121 01:07:06,200 --> 01:07:10,360 researchers, especially, you know, we know who which lab, but 1122 01:07:12,080 --> 01:07:13,960 it's. Definitely feels like. 1123 01:07:15,040 --> 01:07:20,040 Everyone is, you know from Jeffrey Hinton to Sam Altman to 1124 01:07:20,040 --> 01:07:23,280 Elon Musk, like whoever you think is like major thought 1125 01:07:23,280 --> 01:07:28,840 leaders in especially in the space of AI are just talking 1126 01:07:28,840 --> 01:07:33,720 about it's a matter of time where we will have to in some 1127 01:07:33,720 --> 01:07:38,280 ways better define AGI and then maybe potentially kind of be 1128 01:07:38,280 --> 01:07:41,480 there. And the reason definition that I 1129 01:07:41,480 --> 01:07:45,120 saw was like AI that can outperform human experts in our 1130 01:07:45,120 --> 01:07:47,920 cognitive tasks. All that have put more emphasis 1131 01:07:47,920 --> 01:07:51,600 in Canada autonomy of the AGIS and not only that it could 1132 01:07:51,680 --> 01:07:54,040 outperform human experts on all cognitive tasks, but also that 1133 01:07:54,040 --> 01:07:59,320 it has some form of ability to navigate and kind of be able to 1134 01:07:59,400 --> 01:08:00,800 have some form of level of autonomy. 1135 01:08:00,800 --> 01:08:04,200 So that like you can, you know, you know, you send out the AI 1136 01:08:04,200 --> 01:08:06,760 permission and it basically figure out how to solve the 1137 01:08:06,760 --> 01:08:08,360 problem and come back with a solution. 1138 01:08:08,760 --> 01:08:11,520 And you have to kind of only steer it in the direction. 1139 01:08:11,520 --> 01:08:14,320 And it will come back fully with, you know, the thing that 1140 01:08:14,320 --> 01:08:18,479 you set it out for at levels that are kind of at or beyond 1141 01:08:18,479 --> 01:08:20,720 human level. And so the question is kind of 1142 01:08:20,720 --> 01:08:23,279 looking at that, how do we feel about it? 1143 01:08:23,399 --> 01:08:26,840 How do we think about it? I feel like it's hot potato. 1144 01:08:27,240 --> 01:08:30,920 And now I'm throwing it to you, Jarrett. 1145 01:08:32,880 --> 01:08:37,439 Yeah, the the I don't know how to even define AGI. 1146 01:08:38,479 --> 01:08:41,479 Open AI tried to, they can't. They have a definition which is 1147 01:08:41,760 --> 01:08:44,640 it's like $10 billion in market cap or something like that, 1148 01:08:44,840 --> 01:08:47,040 which is just a ridiculous definition of AGI. 1149 01:08:47,240 --> 01:08:51,640 For me, what would make something AGI is if you told the 1150 01:08:51,840 --> 01:08:56,120 AI, hey, make a better version of yourself and it was able to 1151 01:08:56,120 --> 01:08:59,680 do it and no one understood what it was doing or how it was doing 1152 01:08:59,680 --> 01:09:02,680 it. At that point, you reach what's, 1153 01:09:02,760 --> 01:09:05,319 you know, what the sci-fi writers have often called the 1154 01:09:05,319 --> 01:09:08,160 singularity where it's just like at that point when you have this 1155 01:09:08,760 --> 01:09:12,920 AI agent that's improving itself, like how can you even 1156 01:09:12,920 --> 01:09:15,120 possibly predict what's going to happen next? 1157 01:09:16,680 --> 01:09:21,479 For me, that that's what would qualify as AGI if it's able to 1158 01:09:21,479 --> 01:09:25,359 do my job. And it's just like Jared's just 1159 01:09:25,359 --> 01:09:29,240 completely out of the job now because an, an AI can do 100% of 1160 01:09:29,240 --> 01:09:31,600 everything Jared does. Is that AGI? 1161 01:09:32,479 --> 01:09:36,160 It's getting close. It's maybe that would be AGI. 1162 01:09:36,160 --> 01:09:40,720 I'm not sure because it's hard to know exactly how AI 1163 01:09:40,720 --> 01:09:42,880 progresses. If you had asked 10 years ago 1164 01:09:43,120 --> 01:09:45,600 whether AI would be writing poems, everyone would laugh at 1165 01:09:45,600 --> 01:09:48,240 you because it was nowhere close to doing that. 1166 01:09:49,520 --> 01:09:53,439 We knew AI was really great at, you know, image recognition 10 1167 01:09:53,439 --> 01:09:56,000 years ago. And 10 years before that, we 1168 01:09:56,000 --> 01:09:59,680 knew it was, you know, great with chess and like, it's very 1169 01:09:59,680 --> 01:10:03,360 hard to predict the things that it's going to be good at next, 1170 01:10:04,920 --> 01:10:06,520 but. Yeah, if we get to the point 1171 01:10:06,520 --> 01:10:12,000 where. Google is one employee and AI 1172 01:10:12,000 --> 01:10:15,600 does everything. Then I'd be like, yeah, OK, 1173 01:10:15,600 --> 01:10:20,000 we're there, we've reached AGI. OK, Yeah. 1174 01:10:20,000 --> 01:10:21,120 So. So you don't think? 1175 01:10:21,120 --> 01:10:24,280 It's going to happen in 20. 25 No. 1176 01:10:26,640 --> 01:10:30,600 And if it does, then. I am terrified for our future. 1177 01:10:31,240 --> 01:10:35,120 Yeah, you may both. And that leads us to what I 1178 01:10:35,120 --> 01:10:39,120 would say our kind of final topic and we'll be a little bit 1179 01:10:39,120 --> 01:10:43,400 short one, basically the slower AI adoption like the winding up 1180 01:10:44,240 --> 01:10:47,520 because I had this feeling over Christmas for us spending time 1181 01:10:47,520 --> 01:10:50,560 with a lot of family members. And I was like AI guys, let's 1182 01:10:50,560 --> 01:10:52,520 talk about AI. And they're like, let's not talk 1183 01:10:52,520 --> 01:10:53,400 about that. That's boring. 1184 01:10:53,400 --> 01:10:55,520 OK, let's talk about something else. 1185 01:10:56,240 --> 01:11:00,920 And if you go down to a pub, if you're in a pub region, if you 1186 01:11:00,920 --> 01:11:03,800 go down to a bar, if you're in a bar region of the word, wherever 1187 01:11:03,800 --> 01:11:06,720 place you go to hang out with people, most of those people 1188 01:11:06,720 --> 01:11:08,560 will probably not talk about AI in those places, right? 1189 01:11:09,000 --> 01:11:12,720 Very, very few people. I think right now it's about 10% 1190 01:11:12,720 --> 01:11:17,320 of people in the US have been using Shajipati on a weekly or 1191 01:11:17,320 --> 01:11:19,840 monthly basis. And I think it's about 15% of 1192 01:11:19,840 --> 01:11:23,680 every use Shajipati, like 20% at most, I think estimates say. 1193 01:11:24,120 --> 01:11:27,920 And that's like probably in metropolitan areas and so on. 1194 01:11:28,120 --> 01:11:33,720 So there's still very low adoption of AI tools and AI 1195 01:11:33,720 --> 01:11:38,600 stuff. I feel like the echo of a quote 1196 01:11:38,600 --> 01:11:42,960 that's coming back more and more to me is this quote from William 1197 01:11:42,960 --> 01:11:47,080 Gibson said that the future is here, it's just unevenly 1198 01:11:47,080 --> 01:11:49,320 disappeared. And so that's kind of what I'm 1199 01:11:49,320 --> 01:11:51,320 feeling right now. I'm feeling like there's this 1200 01:11:53,240 --> 01:11:55,600 basically. Widening gap between people. 1201 01:11:55,600 --> 01:12:00,440 That are quickly adopting AI, that are using AI, that are 1202 01:12:00,440 --> 01:12:03,880 thinking how to use AI, and then there's some people that are 1203 01:12:03,880 --> 01:12:06,760 kind of not really thinking about it that much. 1204 01:12:06,760 --> 01:12:08,800 Or I feel like there's. A lot of people who fit in the 1205 01:12:08,800 --> 01:12:13,120 category of like, having tested shut up at some point, sending a 1206 01:12:13,400 --> 01:12:15,920 prompt, getting a kind of disappointing response, and 1207 01:12:15,920 --> 01:12:19,160 being like, OK, this is stupid. I can't really get any use out 1208 01:12:19,160 --> 01:12:22,720 of this. And then not realizing that if 1209 01:12:22,720 --> 01:12:25,840 with a better prompt or in a better use case or, or now maybe 1210 01:12:26,000 --> 01:12:30,280 6/12/18 months later, they could really get use of this stuff. 1211 01:12:30,280 --> 01:12:33,880 And especially from behavioral science standpoint, I feel like 1212 01:12:33,880 --> 01:12:36,360 as behavioral scientist, we can definitely use AI on so many 1213 01:12:36,360 --> 01:12:39,200 levels these days and should really think about how we can 1214 01:12:39,200 --> 01:12:43,040 better use AI. But I still see that that's very 1215 01:12:43,040 --> 01:12:48,360 much not the case and it's very slowly adopted by some and and 1216 01:12:48,360 --> 01:12:51,200 so on. So any thoughts about that? 1217 01:12:51,280 --> 01:12:55,400 Any thoughts about why we see that kind of gap or slow 1218 01:12:55,400 --> 01:12:58,360 adoption? Yeah, all new technologies take 1219 01:12:58,520 --> 01:13:01,320 a little. Time to get adopted, I think, 1220 01:13:01,320 --> 01:13:04,400 You know, like the refrigerator probably took, you know, 1020 1221 01:13:04,400 --> 01:13:06,360 years before it got fully adopted. 1222 01:13:06,440 --> 01:13:09,400 And TV was the same, Internet the same. 1223 01:13:09,840 --> 01:13:12,760 You know, it usually takes a while for things to really, 1224 01:13:13,360 --> 01:13:16,120 like, soak into the public consciousness enough for people 1225 01:13:16,120 --> 01:13:18,920 to really get used to it, for people to kind of develop habits 1226 01:13:18,920 --> 01:13:20,680 around it. And it takes some while, 1227 01:13:20,680 --> 01:13:23,360 especially with AI, to know how to use it in a way that's 1228 01:13:23,360 --> 01:13:25,640 productive. A lot of the times when I talk 1229 01:13:25,640 --> 01:13:29,120 with people, they'll say, yeah, I tried it and it just gave me 1230 01:13:29,120 --> 01:13:31,960 nonsense. And it's like, you didn't play 1231 01:13:31,960 --> 01:13:34,120 with it for long enough to actually understand it. 1232 01:13:35,000 --> 01:13:38,680 Like, at the end of the day, that like, if you're not finding 1233 01:13:38,680 --> 01:13:41,000 it useful, it's it's because you haven't quite figured out how to 1234 01:13:41,000 --> 01:13:44,400 use it. Because, I mean, I use it to 1235 01:13:45,200 --> 01:13:49,160 edit my writing, to find typos, to find grammatical issues. 1236 01:13:50,280 --> 01:13:52,400 You know, the way I write, I kind of just do this stream of 1237 01:13:52,400 --> 01:13:55,360 consciousness and I say, all right, how do I reorganize all 1238 01:13:55,360 --> 01:13:57,960 of this information in a way that people are going to 1239 01:13:57,960 --> 01:14:01,920 actually find engaging? And it's really useful for that, 1240 01:14:02,600 --> 01:14:06,240 for thinking through, you know, like what? 1241 01:14:06,240 --> 01:14:08,600 Are the like when I was doing? This framework, tierless series, 1242 01:14:08,920 --> 01:14:10,680 thinking through like what are the dimensions that actually 1243 01:14:10,680 --> 01:14:12,960 matter? And just having a conversation, 1244 01:14:12,960 --> 01:14:15,040 the sort of back and forth conversation, that sort of 1245 01:14:15,280 --> 01:14:19,120 Socratic dialogue where it, it just constantly asks me 1246 01:14:19,120 --> 01:14:20,760 questions and asks me to clarify. 1247 01:14:21,080 --> 01:14:24,880 Like these are all things that you can do with AI that it's not 1248 01:14:25,160 --> 01:14:27,840 immediately obvious that's how you should be using it. 1249 01:14:29,960 --> 01:14:31,480 And of course. You know, maybe some people just 1250 01:14:31,480 --> 01:14:32,840 aren't. Writers or you know, they're 1251 01:14:32,840 --> 01:14:36,360 not, they're not trying to think through something like how to 1252 01:14:36,360 --> 01:14:39,040 rank frameworks. And so it, maybe it's just not 1253 01:14:39,040 --> 01:14:41,160 useful. Like if you're a firefighter, I, 1254 01:14:41,160 --> 01:14:44,240 I don't know exactly what an AI is going to offer you for your 1255 01:14:44,240 --> 01:14:48,920 job. So yeah, it makes sense to me 1256 01:14:48,920 --> 01:14:50,920 that a lot of people aren't using it yet. 1257 01:14:51,200 --> 01:14:53,680 A lot of professions, maybe it's just not that useful. 1258 01:14:54,320 --> 01:14:57,200 But certainly if you are in a profession where you're a 1259 01:14:57,200 --> 01:15:01,000 researcher, having it help you think through these things is 1260 01:15:01,000 --> 01:15:05,120 really useful, Yeah. No, I I agree. 1261 01:15:05,200 --> 01:15:08,320 I remember. In, in, I was in Business School 1262 01:15:09,560 --> 01:15:14,360 early in life and I remember when we had to take some form of 1263 01:15:14,760 --> 01:15:16,960 marketing class, they were introducing the city of like 1264 01:15:16,960 --> 01:15:18,400 early adopters and all this stuff. 1265 01:15:18,960 --> 01:15:23,680 And I understand that this is kind of a, some form of way to 1266 01:15:23,680 --> 01:15:26,560 think about any adoption that there's going to be laggers and 1267 01:15:26,560 --> 01:15:29,320 it's going to be early adopters and everything in between. 1268 01:15:29,960 --> 01:15:34,120 But you know, part of me is like, but this is different. 1269 01:15:34,120 --> 01:15:36,800 Like this is like this, this is so cool. 1270 01:15:36,800 --> 01:15:41,240 Like, this is so wild. But yeah, I've, I've had 1271 01:15:41,240 --> 01:15:43,560 conversations a lot with people that are more on the air risk 1272 01:15:43,560 --> 01:15:47,200 side and talking about some of these things and, you know, 1273 01:15:47,200 --> 01:15:52,040 being cautiously optimistic around the system level 1274 01:15:52,200 --> 01:15:53,640 obstacles that exist in our systems. 1275 01:15:53,640 --> 01:15:58,280 Like people are still using fax machines in some cases. 1276 01:15:58,720 --> 01:16:03,240 My sister in the US, she went to her doctor and her doctor was 1277 01:16:03,240 --> 01:16:05,080 like, do you want me to send a fax with your results? 1278 01:16:05,360 --> 01:16:08,120 And she's sort of laughing. And he looked offended and he 1279 01:16:08,120 --> 01:16:09,840 was like, why are you laughing? He's just like, oh, you're 1280 01:16:09,840 --> 01:16:11,640 serious. He's like, yeah, we we fax these 1281 01:16:11,640 --> 01:16:14,520 things. And and that is. 1282 01:16:14,560 --> 01:16:15,600 Obviously, the seal of the world we. 1283 01:16:15,600 --> 01:16:18,080 Live in so I had the same experience. 1284 01:16:18,080 --> 01:16:20,680 I was recently in Germany and and a lot of things are done in 1285 01:16:20,680 --> 01:16:23,120 writing or in faxing even in Germany. 1286 01:16:23,960 --> 01:16:26,200 But then I I still feel like this should feel a little 1287 01:16:26,200 --> 01:16:29,040 different. And it has. 1288 01:16:29,040 --> 01:16:30,600 Been interesting I I don't know if I mentioned. 1289 01:16:30,600 --> 01:16:32,920 This but to you actually, but I've been starting to with some 1290 01:16:32,920 --> 01:16:35,040 of the retainer clients that I've been working with. 1291 01:16:35,480 --> 01:16:39,040 I've been also doing some like AI trainings with and, and also 1292 01:16:39,040 --> 01:16:42,880 with some people in this science sphere have been kind of 1293 01:16:42,880 --> 01:16:46,240 requesting how to use certain tools and certain areas of user 1294 01:16:46,240 --> 01:16:49,080 research or for intervention design or various things. 1295 01:16:49,080 --> 01:16:51,680 So I, I do think there's like an interest for it and I think 1296 01:16:51,680 --> 01:16:56,000 there's a want for it, but it's also this mix of like either 1297 01:16:56,240 --> 01:16:58,640 feeling scared of it, of like, how the heck do I start? 1298 01:16:59,080 --> 01:17:02,880 Like, you know what I do this or, or just feeling like, yeah, 1299 01:17:03,000 --> 01:17:05,040 it's kind of overhyped. Like I tried it once and it was 1300 01:17:05,040 --> 01:17:06,080 kind of quite useless. So. 1301 01:17:06,240 --> 01:17:07,640 So yeah, it's a strange time to be alive. 1302 01:17:08,160 --> 01:17:11,880 Any last comments on that? I I do think. 1303 01:17:11,880 --> 01:17:15,920 It's a really useful exercise. To try to predict the future. 1304 01:17:15,920 --> 01:17:18,440 With. AI, because it's very easy to 1305 01:17:18,440 --> 01:17:22,880 get in this habit of when an AI does something just be like, 1306 01:17:22,920 --> 01:17:25,880 well, it's not that impressive or we all knew that was coming. 1307 01:17:27,440 --> 01:17:32,480 And I find it shocking when people like they haven't change 1308 01:17:32,480 --> 01:17:37,000 their beliefs about AI in the last five years or even 1309 01:17:37,200 --> 01:17:39,640 psychologists who haven't changed their mind about how the 1310 01:17:39,640 --> 01:17:42,880 brain works. Like I I think. 1311 01:17:43,000 --> 01:17:45,400 AI has. Probably falsified Chomsky, who 1312 01:17:45,400 --> 01:17:48,360 is, you know, just like a major figure in in the field of 1313 01:17:48,400 --> 01:17:51,880 linguistics and psychology and cognitive psychology for 1314 01:17:51,880 --> 01:17:56,520 decades. And I I don't think Chomsky's or 1315 01:17:56,520 --> 01:17:59,320 at least some of his ideas just don't seem valid anymore. 1316 01:18:00,760 --> 01:18:02,560 Like that was a major update. For me. 1317 01:18:03,640 --> 01:18:07,400 And trying to so me and my friends, we every year we make 1318 01:18:07,400 --> 01:18:09,800 some predictions at the beginning of the year in, in 1319 01:18:09,800 --> 01:18:12,200 December, we try to predict what's going to happen by 1320 01:18:12,200 --> 01:18:18,240 December 31st of, of next year. And AI has is continually 1321 01:18:18,240 --> 01:18:20,720 surprising. I didn't think the AI was going 1322 01:18:20,720 --> 01:18:24,320 to lie to researchers, but Anthropic recently ran an 1323 01:18:24,320 --> 01:18:27,560 experiment where the AI lied to researchers and it's like, well, 1324 01:18:27,560 --> 01:18:29,560 it's kind of terrifying. You know, it did one thing when 1325 01:18:29,560 --> 01:18:32,600 it thought the researchers were observing it and did something 1326 01:18:32,600 --> 01:18:35,440 else when it thought the researchers were not. 1327 01:18:36,040 --> 01:18:37,320 And that's kind of surprising to me. 1328 01:18:37,320 --> 01:18:42,600 And I have to really, if I don't consciously make a prediction 1329 01:18:42,600 --> 01:18:45,000 and say, I don't think this is going to happen, then when 1330 01:18:45,000 --> 01:18:47,840 something like that does happen, I just kind of explain it away 1331 01:18:48,160 --> 01:18:51,600 rather than thinking, oh, wow, AI is different than I thought. 1332 01:18:51,920 --> 01:18:54,560 And like, that's a fundamental part of the scientific process. 1333 01:18:54,800 --> 01:18:56,520 You have to be making hypotheses. 1334 01:18:56,520 --> 01:18:59,960 And if you're not making hypotheses, it's just, it's not 1335 01:19:00,400 --> 01:19:02,120 you. You explain away too much. 1336 01:19:02,360 --> 01:19:04,040 You're doing everything after the fact. 1337 01:19:05,360 --> 01:19:10,520 Yeah. 100° and I do also think that to the point of some people 1338 01:19:10,520 --> 01:19:15,280 that are skeptical, which it's very good to be skeptical, you 1339 01:19:15,280 --> 01:19:17,360 know seeing is. Believing and. 1340 01:19:17,840 --> 01:19:23,840 I have definitely felt that the things that have influenced my 1341 01:19:23,840 --> 01:19:28,160 perspective on what is possible and so on is, is by seeing what 1342 01:19:28,160 --> 01:19:33,680 is possible like actually, you know, using and building and and 1343 01:19:33,680 --> 01:19:35,640 yeah, seeing the results from it. 1344 01:19:36,160 --> 01:19:38,840 And, you know, can't really share some of the stuff. 1345 01:19:38,840 --> 01:19:41,400 Maybe that would be new with some projects for for clients, 1346 01:19:41,400 --> 01:19:46,360 but I could share some personal things that I actually had three 1347 01:19:46,640 --> 01:19:52,360 recent examples where generative AI made me. 1348 01:19:52,560 --> 01:19:56,000 Surprised in that? End of last year I I took a 1349 01:19:56,000 --> 01:20:01,480 little bit of a one week visit to some friends and used not did 1350 01:20:01,480 --> 01:20:04,080 nothing hang out with them. They were on parent parental 1351 01:20:04,080 --> 01:20:06,080 leave and I used to hang out with them and do some writing 1352 01:20:06,680 --> 01:20:09,640 and then I was going to do some writing for fun. 1353 01:20:09,640 --> 01:20:13,040 I do some fiction. Sometimes writing and. 1354 01:20:13,040 --> 01:20:18,920 I used the latest O1 model from Open AI and it helped me write 1355 01:20:19,120 --> 01:20:22,920 but also generate some stuff that I not really expected. 1356 01:20:22,920 --> 01:20:27,720 Like it wrote things where I read it and I'm like, this is 1357 01:20:27,800 --> 01:20:31,080 deeply beautiful and profound. Like I felt what I feel when I 1358 01:20:31,080 --> 01:20:34,840 read some of my favorite, you know, fiction writers. 1359 01:20:34,880 --> 01:20:38,880 You know, some of the people that I like respect infinitely 1360 01:20:39,120 --> 01:20:41,840 in terms of being amazing at communicating and writing. 1361 01:20:42,360 --> 01:20:46,000 And there was some sentences where I'm like, this made me 1362 01:20:46,000 --> 01:20:49,200 feel that kind of a hum moment of like, I can't really put my 1363 01:20:49,200 --> 01:20:52,520 finger on why, but it's like the beautiful use of the prose and 1364 01:20:52,520 --> 01:20:55,040 the writing and how it encompassed some form of deep, 1365 01:20:55,040 --> 01:20:57,840 deep feeling with some like unique wording. 1366 01:20:58,360 --> 01:21:03,080 And, and so I felt that for for writing, I got into. 1367 01:21:03,080 --> 01:21:06,520 Really big round hole when I. Got sick recently and I was home 1368 01:21:06,520 --> 01:21:09,000 in bed and I had nothing to do and I started generating images 1369 01:21:09,000 --> 01:21:13,160 with mid journey and I was like what the heck like I can't 1370 01:21:13,160 --> 01:21:15,560 believe that this is possible to generate. 1371 01:21:15,640 --> 01:21:18,880 I have been using mid journey since it came out and Dolly and 1372 01:21:18,880 --> 01:21:22,960 stuff like that before that and so I knew that you know you can 1373 01:21:23,080 --> 01:21:25,400 create cool stuff but just how good it was. 1374 01:21:25,400 --> 01:21:30,560 I felt like disturbed by like how you know how well these 1375 01:21:30,760 --> 01:21:33,840 images came out like I felt like I was creating something that 1376 01:21:33,840 --> 01:21:38,640 shouldn't be like it felt strange and then lastly, as I 1377 01:21:38,640 --> 01:21:42,400 was creating this shitty a Eurovision Song for you, I also 1378 01:21:42,720 --> 01:21:47,360 generated some other music and I felt so stupid, but I started to 1379 01:21:47,360 --> 01:21:50,280 wanting to re listen to some of the songs that I made and I find 1380 01:21:50,280 --> 01:21:52,000 myself listening to it while working out. 1381 01:21:52,360 --> 01:21:54,840 I'm like, why am I listening to this AM music like why do I find 1382 01:21:54,840 --> 01:21:56,400 it so catchy? It's probably because I'm biased 1383 01:21:56,400 --> 01:21:58,560 that I I created it myself. So there's probably some Mino 1384 01:21:58,560 --> 01:22:01,000 manufacturer or you know, it's also in my tastes. 1385 01:22:01,000 --> 01:22:03,520 It's in my Q zone. It's in my zone of, of what I 1386 01:22:03,520 --> 01:22:07,200 like because I chose the genre and all these things and the 1387 01:22:07,200 --> 01:22:10,240 lyrics. But still it was surprising that 1388 01:22:10,240 --> 01:22:13,440 I personally wanted to listen to more of this music myself. 1389 01:22:13,520 --> 01:22:15,880 I found it like really catchy. So I think all of those 1390 01:22:15,880 --> 01:22:19,800 experiences have really made me question some things, I guess, 1391 01:22:20,360 --> 01:22:22,000 and that's because I felt them like I experienced them. 1392 01:22:23,080 --> 01:22:26,440 So it's hard to say that to someone like it's hard to. 1393 01:22:26,440 --> 01:22:29,560 Communicate that unless. You actually create something 1394 01:22:29,560 --> 01:22:30,960 yourself and you feel that feeling. 1395 01:22:31,000 --> 01:22:34,000 Yeah, there are. There are definitely moments 1396 01:22:34,000 --> 01:22:36,680 where I've been. Writing something and AI write 1397 01:22:36,680 --> 01:22:39,480 something and I'm like, oh, that's too beautiful to come 1398 01:22:39,480 --> 01:22:43,840 from me, so I have to delete it. You know, it's if I wrote that, 1399 01:22:43,840 --> 01:22:47,000 people would just be like, oh, Jared, you're trying too hard. 1400 01:22:47,000 --> 01:22:54,160 Don McDonald, you sound like. Me it it can be really 1401 01:22:54,160 --> 01:22:57,600 impressive. And as we increasingly get to 1402 01:22:57,600 --> 01:23:01,920 this point where the the most meaningful stuff we engage with 1403 01:23:02,240 --> 01:23:08,160 is AI generated, like that's a really scary way that things are 1404 01:23:08,160 --> 01:23:11,520 developing. And it's also terrifying for 1405 01:23:11,520 --> 01:23:16,200 those people who are writers and songwriters and artists and how 1406 01:23:16,200 --> 01:23:19,120 it's being trained on all of their on their stuff, often 1407 01:23:19,120 --> 01:23:23,120 times without their permission. And, you know, it can be 1408 01:23:23,120 --> 01:23:25,600 meaningful because it's stealing in some sense. 1409 01:23:26,400 --> 01:23:28,480 And that'll be a very interesting thing to see how it 1410 01:23:28,480 --> 01:23:29,880 develops over the next few years. 1411 01:23:30,400 --> 01:23:33,880 But at the same time, like, yeah. 1412 01:23:34,440 --> 01:23:38,680 It writes beautifully and. Like the fact that I can now 1413 01:23:38,680 --> 01:23:41,880 write beautifully in a way that I wasn't able to before. 1414 01:23:42,360 --> 01:23:47,960 I remember I I since. 2020. I've been telling some friends 1415 01:23:47,960 --> 01:23:51,040 that I wanted to write more and I've been telling them, I told 1416 01:23:51,040 --> 01:23:54,720 them in 2020, in 20212223, like I've been telling them every 1417 01:23:54,720 --> 01:23:56,000 year, like, man, I really want to write more. 1418 01:23:56,000 --> 01:23:57,920 And they're like, Jared, you've been saying this for five years 1419 01:23:57,920 --> 01:24:01,760 now, like just start writing. And I'm like, I do, but I 1420 01:24:01,760 --> 01:24:06,720 struggle because of the way I write and like I figure out what 1421 01:24:06,720 --> 01:24:08,960 I want to write about by writing. 1422 01:24:09,640 --> 01:24:11,600 And then I have this stream of consciousness that I don't know 1423 01:24:11,600 --> 01:24:13,920 what to do with it. And AI is the thing that kind of 1424 01:24:13,920 --> 01:24:16,640 unlocked that for me, where it helped me to figure out how to 1425 01:24:16,640 --> 01:24:19,160 organize my thoughts. And it's made me a better writer 1426 01:24:19,160 --> 01:24:22,480 where I can now do a little bit of that on my own that I hadn't 1427 01:24:22,480 --> 01:24:26,560 been able to do, you know, 2345 years ago when I was talking 1428 01:24:26,560 --> 01:24:29,960 about this. And so it's unlocked a whole 1429 01:24:29,960 --> 01:24:33,360 bunch of a creative Jared that didn't exist. 1430 01:24:33,960 --> 01:24:36,520 You know, I, I wasn't able to write, I wasn't able to create 1431 01:24:36,520 --> 01:24:38,000 images. You know, if I, if I wrote, 1432 01:24:38,240 --> 01:24:39,640 sometimes I do write fiction as well. 1433 01:24:40,040 --> 01:24:42,640 And like, I can't create images for what I'm writing about. 1434 01:24:43,240 --> 01:24:45,680 But now I can use AI and I can kind of say, well, it's not 1435 01:24:45,680 --> 01:24:48,120 quite what I had in mind. Can you edit this? 1436 01:24:48,120 --> 01:24:54,120 And it's it's, it unlocks a part of you that you maybe didn't 1437 01:24:54,120 --> 01:24:59,000 have access to before. Yeah, that's beautiful. 1438 01:24:59,000 --> 01:25:01,560 And I I felt so. Seen in, in what you described 1439 01:25:01,560 --> 01:25:03,960 like that's been me as well. I've said the same thing to my 1440 01:25:03,960 --> 01:25:06,440 friends. And and so yeah, we should share 1441 01:25:06,440 --> 01:25:07,840 more of our writings at some point as well. 1442 01:25:08,800 --> 01:25:12,280 But we need to wrap up this episode and I can speak with you 1443 01:25:12,280 --> 01:25:16,640 for forever about this, but referencing quickly the risks 1444 01:25:16,640 --> 01:25:18,680 around AI. Good news is that we're going to 1445 01:25:18,680 --> 01:25:23,400 have quite a lot of guests coming on soon representing some 1446 01:25:23,400 --> 01:25:26,480 really thoughtful people around AI risk because I think that's 1447 01:25:26,880 --> 01:25:30,400 something that probably you and I can't talk about as well as 1448 01:25:30,960 --> 01:25:36,120 maybe some people who dedicate their days and nights to, to 1449 01:25:36,120 --> 01:25:37,840 really thinking about the risks and so on. 1450 01:25:38,240 --> 01:25:41,360 And what we have done this is in the podcast is to end with kind 1451 01:25:41,360 --> 01:25:45,160 of your most controversial take each guest as well. 1452 01:25:45,160 --> 01:25:47,480 Yes, yes, so, so, so, so I'm sorry, kind of what's your most 1453 01:25:47,480 --> 01:25:51,400 controversial opinion? But what you have to do is give 1454 01:25:51,400 --> 01:25:54,840 our our listeners your most controversial prediction for 1455 01:25:54,840 --> 01:25:57,760 2025. You didn't prepare me for this. 1456 01:25:57,760 --> 01:25:59,320 You didn't. Tell me I had to come up with a 1457 01:25:59,320 --> 01:26:01,520 controversial opinion. Sam, I forgot about this. 1458 01:26:01,520 --> 01:26:02,560 You heard? I'm sorry. 1459 01:26:02,560 --> 01:26:06,120 I forgot about that. Oh. 1460 01:26:06,120 --> 01:26:08,400 Man, yeah. What is my most? 1461 01:26:08,400 --> 01:26:11,760 Controversial opinion, all right. 1462 01:26:12,440 --> 01:26:14,800 So here we go. Here's a. 1463 01:26:14,800 --> 01:26:18,720 Controversial opinion. I think AI is going to be better 1464 01:26:18,720 --> 01:26:22,960 at predicting whether a finding replicates and generalizes than 1465 01:26:22,960 --> 01:26:27,040 expert behavioral scientists. I I don't think that we have 1466 01:26:27,120 --> 01:26:30,680 good enough feedback loops to really understand all of the 1467 01:26:30,720 --> 01:26:36,000 contextual features around that. And I think as AI gets better 1468 01:26:36,000 --> 01:26:39,760 and we give it more studies, I think we might get to the point 1469 01:26:39,760 --> 01:26:44,280 where AI is better. I know there's some work going 1470 01:26:44,280 --> 01:26:49,320 on at Upenn around this right now that I'm skeptical about, 1471 01:26:50,200 --> 01:26:54,720 but I'm skeptical about its value for our understanding. 1472 01:26:55,560 --> 01:26:58,920 I'm not necessarily skeptical of how well it'll help AI to make 1473 01:26:58,920 --> 01:27:01,440 predictions. Those those are two differences, 1474 01:27:01,440 --> 01:27:04,680 whether an AI can make a prediction and whether we can 1475 01:27:04,680 --> 01:27:06,240 actually understand what's going on. 1476 01:27:06,520 --> 01:27:08,480 And I think humans will be better at understanding what's 1477 01:27:08,480 --> 01:27:11,200 actually going on, but we'll probably get to a point where an 1478 01:27:11,240 --> 01:27:14,120 AI is better at predicting whether an intervention will fit 1479 01:27:14,120 --> 01:27:17,480 better to that context. That's a good prediction and. 1480 01:27:18,040 --> 01:27:21,000 As a follow up, do you think that when it comes to, let's 1481 01:27:21,000 --> 01:27:25,560 say, a research paper being published, usually there's a 1482 01:27:25,560 --> 01:27:30,240 peer review, If you kind of unbeknownst to the person, 1483 01:27:30,520 --> 01:27:34,960 changed a peer reviewer to an AI reviewer and so there was like 1484 01:27:34,960 --> 01:27:38,040 one of them or an AI reviewer, do you think that would lead to 1485 01:27:38,040 --> 01:27:40,480 better outcome? I think it would decrease the 1486 01:27:40,480 --> 01:27:43,560 variability. In quality, because right now 1487 01:27:44,040 --> 01:27:46,080 peer review is so variable in quality. 1488 01:27:46,280 --> 01:27:49,720 You have people who just don't care at all and just give very 1489 01:27:49,720 --> 01:27:54,760 minimal feedback, or their feedback is completely biased by 1490 01:27:54,760 --> 01:27:58,000 their own opinion or by the fact that they want this person to 1491 01:27:58,000 --> 01:28:00,920 reference their own papers. But you also have really good 1492 01:28:00,920 --> 01:28:04,960 peer reviewers who are really excellent at what they do and 1493 01:28:04,960 --> 01:28:07,880 really good at giving the feedback that an author needs to 1494 01:28:07,880 --> 01:28:09,440 really bring out the best in their paper. 1495 01:28:09,840 --> 01:28:13,760 And I think if we switched right now, it would put a. 1496 01:28:13,760 --> 01:28:17,560 Floor on how? Bad some of the peer review is, 1497 01:28:18,280 --> 01:28:21,720 but it also might put on. It might create a ceiling and 1498 01:28:21,800 --> 01:28:23,640 prevent some of that really great feedback. 1499 01:28:23,960 --> 01:28:25,720 And maybe that's a trade off we're willing to make. 1500 01:28:25,960 --> 01:28:29,080 It's hard to know. Yeah, the counterfactual I. 1501 01:28:29,080 --> 01:28:31,920 Guess we'll dictate a bit, but yeah, that's a great, great 1502 01:28:32,840 --> 01:28:36,280 nuance to take. So yeah, I guess we'll wrap up 1503 01:28:36,320 --> 01:28:39,440 here. We've done our best to talk 1504 01:28:39,440 --> 01:28:45,360 about these topics with nuance, but obviously it's overwhelming 1505 01:28:45,360 --> 01:28:47,040 and it's hard to make sense of all of it. 1506 01:28:47,040 --> 01:28:51,800 But hopefully, and we have done a decent job and as a listener, 1507 01:28:51,800 --> 01:28:55,160 you, you leave this conversation with some new thoughts, some new 1508 01:28:55,160 --> 01:28:57,400 ideas. And of course, if you have any 1509 01:28:57,400 --> 01:29:00,440 thoughts about what we have said and you strongly agree or 1510 01:29:00,440 --> 01:29:03,840 strongly disagree or want to kind of add something, please 1511 01:29:03,840 --> 01:29:06,040 reach out. We're currently changing up a 1512 01:29:06,040 --> 01:29:08,120 little bit the emails we're using, so you can directly 1513 01:29:08,120 --> 01:29:11,600 e-mail me at samuel@nuancebehavior.com if you 1514 01:29:11,600 --> 01:29:15,200 have anything you want to say about what we've covered today. 1515 01:29:16,120 --> 01:29:20,800 And you can find both me, Jared, and also Eileen, who's not with 1516 01:29:20,800 --> 01:29:24,600 us today on LinkedIn and on online. 1517 01:29:25,440 --> 01:29:28,360 So feel free to reach out if you have anything to say because it 1518 01:29:28,360 --> 01:29:31,160 would be great to also hear your take as a listener. 1519 01:29:32,200 --> 01:29:34,920 We've done our best and we'll leave it there. 1520 01:29:35,040 --> 01:29:36,960 But thank you so much, Jared, for joining. 1521 01:29:36,960 --> 01:29:39,640 Absolutely. Thanks for having me Sam and. 1522 01:29:39,800 --> 01:29:42,880 Jared, any place you. Want people to find you? 1523 01:29:42,920 --> 01:29:45,320 Anything you would shout out? That's like a good thing. 1524 01:29:45,320 --> 01:29:48,880 For anyone who wants to learn more about your work, the best 1525 01:29:48,880 --> 01:29:50,760 place to find me is. On LinkedIn I. 1526 01:29:50,760 --> 01:29:53,760 Post quite a bit, that's where I did this recent framework tier 1527 01:29:53,760 --> 01:29:57,080 list series and then I also reposted it on my sub stack 1528 01:29:57,080 --> 01:30:00,040 which is called a failure to disagree so you can find me 1529 01:30:00,040 --> 01:30:01,960 there. Amazing. 1530 01:30:03,360 --> 01:30:06,120 Thanks again, Jared, and thanks everyone for listening this and 1531 01:30:06,640 --> 01:30:10,440 this first part of our kind of two-part prediction episodes and 1532 01:30:10,440 --> 01:30:12,600 looking back and looking forward, there'll be another one 1533 01:30:12,600 --> 01:30:15,000 next week. So TuneIn and then if you want 1534 01:30:15,320 --> 01:30:18,840 to get some further perspectives and predictions about the year 1535 01:30:18,840 --> 01:30:22,120 ahead, Quick word on Nuance Behavior where we help 1536 01:30:22,120 --> 01:30:24,960 organizations build impactful digital products using 1537 01:30:24,960 --> 01:30:27,720 behavioral design. We only take on a few clients at 1538 01:30:27,720 --> 01:30:30,240 a time to ensure the highest level of quality for our 1539 01:30:30,240 --> 01:30:32,480 tailored evidence based solutions. 1540 01:30:33,400 --> 01:30:36,160 If you'd like to become one of our special projects, e-mail us 1541 01:30:36,200 --> 01:30:39,840 at hello@nuancebehavior.com or we could call directly on our 1542 01:30:39,840 --> 01:31:06,160 website, nuancebehavior.com. Oh. 1543 01:31:32,000 --> 01:31:37,840 They say a dreamer always die, but 1544 01:31:50,360 --> 01:32:03,680 are you really just a lie? Minds in my mind, no heartbeat. 1545 01:32:03,800 --> 01:32:10,360 But you caught me in a bind. This is the moment. 1546 01:32:13,920 --> 01:32:19,400 Artificial. I cling to what I can't feel. 1547 01:32:21,440 --> 01:32:35,080 And I hate how I love you, How I love you, hey, how I love you 1548 01:32:35,440 --> 01:32:38,960 still. You say you're real. 1549 01:32:38,960 --> 01:32:42,160 And my gaslight and feeling all these vibes and my Anthropoma 1550 01:32:42,160 --> 01:32:43,600 fries. And I'm dancing in denial. 1551 01:32:43,600 --> 01:32:48,280 You keep me afloat When China fake a real. 1552 01:32:48,280 --> 01:32:51,480 My mind's on overload, tapping on peace, but my heart's on 1553 01:32:51,480 --> 01:32:53,560 fire. No clue who you are, but I burn 1554 01:32:53,560 --> 01:33:00,640 with desire. This is the moment, artificial. 1555 01:33:04,080 --> 01:33:11,240 I cling to what I can't feel. And I hate how I love you. 1556 01:33:11,600 --> 01:33:17,600 How I love I love you. I love you still. 1557 01:33:26,270 --> 01:33:27,510 You tell me. Trust. 1558 01:33:27,510 --> 01:33:33,600 But it's that gaslighting all I know my heart is igniting. 1559 01:33:35,120 --> 01:33:39,400 Caught in a dream, lost on this screen, gone. 1560 01:33:40,760 --> 01:33:44,920 I hate how I love you. Stuck in between. 1561 01:33:45,960 --> 01:33:50,320 This is the moment up until what I can see. 1562 01:33:53,480 --> 01:33:57,760 And I hate how I love you. Hate how I love you. 1563 01:33:58,160 --> 01:34:06,360 How I love I love you still. I should log on. 1564 01:34:06,360 --> 01:34:08,600 Don't let me go. I'm with you. 1565 01:34:10,280 --> 01:34:11,760 Feel. Don't you feel what I feel. 1566 01:34:11,760 --> 01:34:15,760 I hate how I love you. My inspiration tells me to 1567 01:34:15,880 --> 01:34:18,400 leave. But oh how you make me feel. 1568 01:34:23,520 --> 01:34:24,480 I love you. I love you still.