1 00:00:03,280 --> 00:00:05,880 Hello and welcome to the Behavioral Design Podcast. 2 00:00:05,960 --> 00:00:08,680 This season we're diving into the intersection of behavioral 3 00:00:08,680 --> 00:00:11,080 science and AI. We want to make sense of the 4 00:00:11,080 --> 00:00:14,080 state of AI, from understanding how humans interact with 5 00:00:14,080 --> 00:00:17,520 intelligent systems to using AI to do behavioral design itself. 6 00:00:17,720 --> 00:00:21,280 I'm Aline Holsworth, a health tech advisor specializing in AI 7 00:00:21,280 --> 00:00:23,880 and product design. Over the past 15 years, I've 8 00:00:23,880 --> 00:00:26,720 been crafting human centered products with behavioral science 9 00:00:26,720 --> 00:00:29,160 at the core. At Apple, I LED Behavioral 10 00:00:29,160 --> 00:00:32,119 Science for Health AI, designing and launching AI powered 11 00:00:32,119 --> 00:00:34,440 features to help users reach their health goals. 12 00:00:35,160 --> 00:00:37,160 And I'm Samuel Sultzer, your second Co host. 13 00:00:37,400 --> 00:00:40,600 I'm a behavioral strategist specializing in hybrid formation 14 00:00:40,600 --> 00:00:43,200 and designing products that drive long term baby change. 15 00:00:43,640 --> 00:00:46,880 I work with leading tech organizations integrating AI to 16 00:00:46,880 --> 00:00:48,240 scale behavioral design for good. 17 00:00:48,800 --> 00:00:52,200 And I'm also the founder of Baby Bites, a dedicated community on 18 00:00:52,440 --> 00:00:56,240 behavioral science and AI. Quick word on Nuance Behavior 19 00:00:56,240 --> 00:00:59,640 where we help organizations build impactful digital products 20 00:00:59,640 --> 00:01:02,640 using behavioral design. We only take on a few clients at 21 00:01:02,640 --> 00:01:05,200 a time to ensure the highest level of quality for our 22 00:01:05,200 --> 00:01:07,400 tailored evidence based solutions. 23 00:01:08,280 --> 00:01:11,080 If you'd like to become one of our special projects, e-mail us 24 00:01:11,120 --> 00:01:14,760 at hello@nuancebehavior.com or we could call directly on our 25 00:01:14,760 --> 00:01:36,080 website, nuancebehavior.com. Hey Aline, I'm curious if you've 26 00:01:36,080 --> 00:01:41,360 played this game that I call the do but not recommend game if you 27 00:01:41,360 --> 00:01:44,960 live with us? It sounds a little bit like you 28 00:01:44,960 --> 00:01:47,800 just made this game up. I I somewhat did. 29 00:01:48,240 --> 00:01:52,080 It reflects a little bit of my own behavior, navigating a world 30 00:01:52,080 --> 00:01:53,880 filled with recommendation systems. 31 00:01:54,280 --> 00:01:58,280 And what I mean here is like I do something, but the game is 32 00:01:58,480 --> 00:02:01,840 for me to ensure that the platform I'm using is not 33 00:02:01,840 --> 00:02:05,200 recommending me things based on what I do and so. 34 00:02:05,680 --> 00:02:07,680 Because they're things that you don't want to do. 35 00:02:08,199 --> 00:02:09,880 They're things that you wish you didn't do. 36 00:02:10,600 --> 00:02:15,680 Yeah, like so on Twitter for example, I would say a lot of my 37 00:02:15,680 --> 00:02:21,800 time is spent kind of reading about facts or things relating 38 00:02:21,800 --> 00:02:26,360 to my interest in football. And so I for football club, I 39 00:02:26,360 --> 00:02:28,800 get some news around that and and so on. 40 00:02:28,800 --> 00:02:33,520 But I've made sure that in the feed I've made it so that I 41 00:02:33,520 --> 00:02:36,440 never recommended any football facts or any any football news 42 00:02:36,440 --> 00:02:39,480 and so on. So I had to act yeah, yeah. 43 00:02:39,480 --> 00:02:42,080 You can use like block word, certain words, you can downvote 44 00:02:42,080 --> 00:02:44,240 certain things, you can like depends on the platform. 45 00:02:44,240 --> 00:02:46,880 You can kind of like say, don't recommend me this, don't 46 00:02:46,880 --> 00:02:50,200 recommend me this. So I usually have to like I have 47 00:02:50,200 --> 00:02:56,240 to go into the various accounts that I kind of still follow to 48 00:02:56,240 --> 00:02:58,680 get the news. So I'm like playing this game of 49 00:02:58,680 --> 00:03:04,200 like getting into to this because I see myself as someone 50 00:03:04,200 --> 00:03:07,000 who's engaging with Twitter in order to stay up to date with 51 00:03:07,000 --> 00:03:09,360 what's happening in AI and behavioral science and so on. 52 00:03:09,640 --> 00:03:12,880 And not to stay on top of what's happening in football, even 53 00:03:12,880 --> 00:03:14,200 though that's what I often times do. 54 00:03:14,600 --> 00:03:16,840 So I'm I'm playing this game and I'm curious if you can relate to 55 00:03:16,840 --> 00:03:19,040 it or? Yeah, in some ways it sounds 56 00:03:19,040 --> 00:03:21,920 like a pretty nice commitment device, right? 57 00:03:21,920 --> 00:03:25,520 Like you're adding all this extra friction to getting the 58 00:03:25,520 --> 00:03:29,440 information about your soccer. I'm just going to say soccer. 59 00:03:32,000 --> 00:03:36,560 And, and maybe you don't consume as much soccer as you would 60 00:03:36,640 --> 00:03:39,360 otherwise, which, you know, maybe it's part of the goal, 61 00:03:39,360 --> 00:03:42,360 maybe it's not. But honestly, when it comes to 62 00:03:42,400 --> 00:03:45,640 Twitter or X itself, I pretty much stopped using it when it 63 00:03:45,640 --> 00:03:48,480 became X because I, the recommendations that I got were, 64 00:03:48,800 --> 00:03:51,800 were so far off from what I was interested in. 65 00:03:52,120 --> 00:03:57,480 I felt like I had for many years really sort of, I don't know, 66 00:03:57,680 --> 00:04:02,200 led the algorithm towards things that I was really interested in. 67 00:04:02,200 --> 00:04:05,120 It was purely academic. It was it just like my 68 00:04:05,120 --> 00:04:09,480 colleagues sharing their new findings and, you know, adding 69 00:04:09,480 --> 00:04:13,840 the TLDRS about it and just just really, really useful summaries 70 00:04:13,840 --> 00:04:16,480 and commentary about what would have been published. 71 00:04:17,959 --> 00:04:20,640 And that at some point all of those disappeared and I just got 72 00:04:20,720 --> 00:04:24,680 political commentary and, and things that I'm just so, so, so 73 00:04:25,160 --> 00:04:28,760 uninterested in or lately I don't, I don't want the, the X 74 00:04:28,760 --> 00:04:32,320 version of that, let's say. But, but going back to your 75 00:04:32,320 --> 00:04:36,800 original question, do I, do I do any of that manipulation? 76 00:04:36,800 --> 00:04:42,840 I actually do a much lamer version of that, which is I just 77 00:04:42,840 --> 00:04:45,760 exhibit self-control in the moment, so when I see. 78 00:04:45,760 --> 00:04:47,840 Something. I do, yeah. 79 00:04:47,840 --> 00:04:52,600 I actively stop myself. I say to myself, I see something 80 00:04:52,600 --> 00:04:57,200 that I that I don't want to be. I don't want that view to be 81 00:04:57,200 --> 00:04:59,560 incorporated into into my algorithm. 82 00:04:59,880 --> 00:05:03,880 I look at it and I say don't do it, that's going to work against 83 00:05:03,880 --> 00:05:06,040 you. And then I actually don't look 84 00:05:06,040 --> 00:05:10,520 at it as much as I want to. I I tell myself, Nope, you're 85 00:05:10,520 --> 00:05:13,120 going to be seeing those stupid videos for the rest of your 86 00:05:13,120 --> 00:05:15,520 life. Yeah, but then you're like, 87 00:05:15,520 --> 00:05:17,800 aware of the consequences of like, actually watching this 88 00:05:17,800 --> 00:05:19,920 video will have some ramifications. 89 00:05:20,560 --> 00:05:23,800 But I don't miss it. I, I think that the difference 90 00:05:23,800 --> 00:05:29,000 is that those, the videos that I'm tempted to look at are not 91 00:05:29,000 --> 00:05:32,240 something that I, I really care about keeping up on. 92 00:05:32,240 --> 00:05:36,400 Whereas for you with your, with your silly sports, you do, you 93 00:05:36,400 --> 00:05:38,440 do care about keeping track of that thing. 94 00:05:38,440 --> 00:05:43,800 For me, it's like parenting videos, you know, like people, 95 00:05:43,880 --> 00:05:45,760 people telling you it's all right, you're going to get 96 00:05:45,760 --> 00:05:47,720 through this and here's why parenting is so hard. 97 00:05:47,720 --> 00:05:51,440 Like you don't really need to see some random stranger with a 98 00:05:51,440 --> 00:05:54,600 million followers telling me that that's not how I want to 99 00:05:54,600 --> 00:05:58,400 spend my time. So yeah, that that has been my 100 00:05:58,400 --> 00:05:59,880 solution. That's fair. 101 00:05:59,880 --> 00:06:02,280 And I guess that's the last point on this from from my, I 102 00:06:02,440 --> 00:06:06,200 guess I would love to say that my motivation was that it's like 103 00:06:06,200 --> 00:06:08,120 some form of commitment advice in a positive way. 104 00:06:08,800 --> 00:06:12,800 But I think it's a little bit of like some form of Dorian Gray 105 00:06:13,560 --> 00:06:18,520 thing in terms of, you know, how in Dorian Gray is this person 106 00:06:18,520 --> 00:06:20,920 who lives a life where he's beautiful, but he has like a 107 00:06:20,960 --> 00:06:23,960 image of himself that becomes ugly and ugly and uglier in 108 00:06:23,960 --> 00:06:27,680 hiding. And I feel like what I don't 109 00:06:27,680 --> 00:06:31,440 want to see is when I open my feed is to be reminded of the 110 00:06:31,480 --> 00:06:33,360 ugliness, like in terms of my behavior. 111 00:06:35,320 --> 00:06:37,520 I don't want to see that I am someone interested in. 112 00:06:37,520 --> 00:06:40,520 Turn the mirror. Away some other things and so I 113 00:06:40,520 --> 00:06:46,640 think kind of ensuring some form of yeah, nice image of myself is 114 00:06:46,640 --> 00:06:50,280 probably why I do it. You're you're sweeping ugly Sam 115 00:06:50,280 --> 00:06:54,840 under the rug, hiding him so that you can believe you. 116 00:06:54,880 --> 00:06:59,480 You can even fool yourself into believing that that this is that 117 00:06:59,480 --> 00:07:03,280 ideal Sam is Sam. Wow, interesting. 118 00:07:04,120 --> 00:07:08,760 All right, cool. I love talking about recommender 119 00:07:08,760 --> 00:07:13,440 systems and I, I also love how the algorithm has just become 120 00:07:13,440 --> 00:07:17,920 part of like general conversation. 121 00:07:17,920 --> 00:07:21,120 Like everyone understands that they're being fed 122 00:07:21,120 --> 00:07:24,680 recommendations based on what they've seen or what they've 123 00:07:24,680 --> 00:07:27,640 done. And, and that, that this is just 124 00:07:27,640 --> 00:07:30,960 a part of everyday language. Like what, what you're doing to 125 00:07:31,080 --> 00:07:34,920 modify, to control or not control your algorithm. 126 00:07:35,000 --> 00:07:37,360 It's this is not even niche language anymore. 127 00:07:38,640 --> 00:07:41,760 No, we're way past that point where we said like, oh, isn't it 128 00:07:41,760 --> 00:07:45,440 creepy that what I do online impacts what I see and stuff 129 00:07:45,440 --> 00:07:49,400 like that. It's just that's just so 2010 or 130 00:07:49,400 --> 00:07:52,200 15 or something. Yeah, pretty much. 131 00:07:53,080 --> 00:07:57,800 Well, and it's funny because even though this, this is has 132 00:07:57,800 --> 00:08:01,720 become so familiar to all of us, what I didn't realize until a 133 00:08:01,720 --> 00:08:06,400 couple years ago was just how complicated these systems are. 134 00:08:06,880 --> 00:08:11,520 And Oh my gosh, until I, I was really like in the weeds of 135 00:08:11,520 --> 00:08:13,320 working with recommender systems. 136 00:08:13,320 --> 00:08:17,000 I just, I just didn't have an appreciation for the math and 137 00:08:17,000 --> 00:08:19,640 the engineering and the the complexity behind them. 138 00:08:20,160 --> 00:08:22,560 You had some, you had some first time experience. 139 00:08:23,200 --> 00:08:25,480 In, in I did. Yeah, yeah, I worked. 140 00:08:25,480 --> 00:08:27,320 I worked quite a bit with Recommender Systems. 141 00:08:28,120 --> 00:08:31,720 I won't say where, but but yeah, I thought it might be useful to 142 00:08:31,720 --> 00:08:35,840 do a very high level overview of, you know, if you like take a 143 00:08:35,840 --> 00:08:39,159 recommender system course that that you always get the, the 144 00:08:39,159 --> 00:08:41,799 sort of these are the types of recommender systems. 145 00:08:41,799 --> 00:08:44,440 I thought I'd do that for us today for our behavioral 146 00:08:44,440 --> 00:08:47,280 scientists audience who you know, probably hasn't come into 147 00:08:47,480 --> 00:08:51,200 into contact with this. But I think it'll be also useful 148 00:08:51,200 --> 00:08:54,600 for our conversation with Carrie where we really get into the 149 00:08:54,600 --> 00:08:58,480 types of data and how preferences are modelled. 150 00:08:58,680 --> 00:09:02,160 So when we, when we think about our recommender systems, one O 151 00:09:02,160 --> 00:09:05,800 1, there are two major categories and how they work. 152 00:09:05,800 --> 00:09:11,160 We have content based filtering, which where each user basically 153 00:09:11,440 --> 00:09:14,400 a profile is created for them based on what they say they're 154 00:09:14,400 --> 00:09:18,000 interested in. So for you, Sam, this would just 155 00:09:18,000 --> 00:09:21,280 be your your academic articles on Twitter. 156 00:09:22,040 --> 00:09:25,360 And then that information is matched to content in order to 157 00:09:25,360 --> 00:09:27,920 provide recommendations to to users. 158 00:09:28,360 --> 00:09:32,640 So there's type 1 and then type 2 is the collaborative 159 00:09:32,640 --> 00:09:35,480 filtering. And this is I think often what 160 00:09:35,480 --> 00:09:38,000 we think of as recommender systems. 161 00:09:38,000 --> 00:09:40,680 And that's because it is more commonly used. 162 00:09:41,320 --> 00:09:46,040 And this is where you, you're really relying on people's 163 00:09:46,040 --> 00:09:48,360 implicit data. So what there's what they're 164 00:09:48,360 --> 00:09:50,080 viewing, what they're clicking on. 165 00:09:50,760 --> 00:09:53,480 And, and this would be if you didn't override the system, this 166 00:09:53,480 --> 00:09:56,480 would be your, your soccer content. 167 00:09:57,600 --> 00:09:59,480 So they're basically inferred preferences. 168 00:09:59,480 --> 00:10:03,840 We, you know the we, we the algorithm, think that you prefer 169 00:10:03,880 --> 00:10:08,440 watching videos of soccer because that's what we see you 170 00:10:08,440 --> 00:10:12,280 secretly doing at night with your flashlight under the 171 00:10:12,280 --> 00:10:13,480 covers. Yeah. 172 00:10:13,480 --> 00:10:17,000 And I guess this is what I would say of all of the different 173 00:10:17,000 --> 00:10:20,560 algorithms and recognition systems, I think TikTok is 174 00:10:20,560 --> 00:10:24,720 probably the one that has become most kind of famous or infamous 175 00:10:24,720 --> 00:10:29,920 for this type of kind of not only relying on kind of what we 176 00:10:29,920 --> 00:10:32,840 click on, but use like the time spent looking at something. 177 00:10:32,920 --> 00:10:38,640 I do feel like the the vernacular of train the 178 00:10:38,640 --> 00:10:42,720 algorithm actually really did come out of the early TikTok 179 00:10:42,720 --> 00:10:46,360 days where people realized it was just so blatant and obvious. 180 00:10:46,360 --> 00:10:49,360 If you watched one video that you know, the you would be 181 00:10:49,360 --> 00:10:53,240 served up another video just like that one right after. 182 00:10:53,240 --> 00:10:55,000 So it was, it's just so blatantly obvious. 183 00:10:55,200 --> 00:10:58,560 I think that's sort of is when it entered our vocabulary 184 00:10:59,560 --> 00:11:02,320 awesome content based and collaborative filtering, two 185 00:11:02,400 --> 00:11:04,960 major categories. Of course most recommender 186 00:11:04,960 --> 00:11:07,560 systems that we have today are some combination of those and we 187 00:11:07,560 --> 00:11:09,680 call those hybrid models. Yeah. 188 00:11:09,680 --> 00:11:14,600 And so those are major types. One thing that it that I think 189 00:11:14,600 --> 00:11:18,680 is maybe interesting about the types as well is just that there 190 00:11:19,040 --> 00:11:22,440 is so much more implicit data available. 191 00:11:22,800 --> 00:11:30,200 And so just as a result of that surplus, you know, you, you kind 192 00:11:30,200 --> 00:11:33,560 of can only use the data that you have. 193 00:11:33,560 --> 00:11:38,800 In some ways that ends up being what companies rely on in order 194 00:11:38,800 --> 00:11:41,160 to make the recommendations, but it also tends to be more 195 00:11:41,160 --> 00:11:43,640 predictive of time spent on the platform. 196 00:11:43,640 --> 00:11:47,240 So if that's what you're you're trying to optimize, which in 197 00:11:47,240 --> 00:11:52,080 many cases unfortunately is you, then you might be sending more 198 00:11:52,160 --> 00:11:55,480 gratifying content that is that's going to keep people, 199 00:11:55,760 --> 00:11:59,400 keep people clicking and viewing and scrolling and so on. 200 00:11:59,840 --> 00:12:01,800 Yeah, I know this is such an interesting subject and I'm so 201 00:12:01,800 --> 00:12:05,160 happy we had a chance to to speak with with Carrie about 202 00:12:05,160 --> 00:12:06,280 this. So maybe, yeah, do you want to 203 00:12:06,280 --> 00:12:08,200 introduce Carrie? I would love to. 204 00:12:08,520 --> 00:12:12,880 So we have a very special guest today, Carrie Morwich. 205 00:12:12,880 --> 00:12:16,960 And for me personally, Carrie is really the person who took what 206 00:12:16,960 --> 00:12:20,160 was purely an academic love of behavioral science and showed me 207 00:12:20,160 --> 00:12:23,680 that that this could actually be a career, not just a passion. 208 00:12:24,080 --> 00:12:27,920 And so Carrie was was really my very first real mentor, the 209 00:12:27,920 --> 00:12:32,000 first person who kind of picked me under his wing and showed me 210 00:12:32,000 --> 00:12:34,280 the way. I met him when I was an intern 211 00:12:34,400 --> 00:12:36,400 actually at Carnegie Mellon way back in. 212 00:12:36,520 --> 00:12:40,240 And I'm not sure I should date myself, but back in 2008. 213 00:12:41,160 --> 00:12:45,040 And I was so excited by the work that that we did together during 214 00:12:45,040 --> 00:12:48,280 this internship that I actually asked him to be an advisor on my 215 00:12:48,280 --> 00:12:49,560 thesis. So he agreed. 216 00:12:49,560 --> 00:12:53,600 We, we get to work together and just the, the amount that I 217 00:12:53,600 --> 00:12:57,240 learned from him in that period is, is unparalleled. 218 00:12:57,640 --> 00:13:02,800 And I, I remember this moment, I think we were eating lunch 219 00:13:03,280 --> 00:13:06,360 somewhere and we were, we were talking about my thesis work and 220 00:13:06,800 --> 00:13:10,360 just just totally out of thin air, he pulled out this, a full 221 00:13:10,360 --> 00:13:14,080 citation for some statistical analysis that we were talking 222 00:13:14,080 --> 00:13:17,120 about. And I was, I was just blown 223 00:13:17,320 --> 00:13:19,120 away. My, I'm just completely 224 00:13:19,120 --> 00:13:21,680 awestruck. Like, like this guy is pure 225 00:13:21,680 --> 00:13:23,920 genius. Like how, how is he doing that? 226 00:13:25,160 --> 00:13:29,040 And yeah, Carrie really taught me what it what I think it takes 227 00:13:29,040 --> 00:13:30,720 to become an expert at something. 228 00:13:31,640 --> 00:13:35,440 So if we Fast forward to today, Carrie is a professor of 229 00:13:35,440 --> 00:13:39,360 marketing at Boston University. Here he he continues to strike 230 00:13:39,360 --> 00:13:42,760 on his students and to research decision making. 231 00:13:43,400 --> 00:13:47,240 Yeah, well, I I first of all, I love this idea of having in your 232 00:13:47,240 --> 00:13:52,800 back pocket a cool citation to to impress people with but but 233 00:13:52,800 --> 00:13:55,840 also I'm interested. So tell us more about what does 234 00:13:55,840 --> 00:13:58,320 it mean here in terms of research decision making in the 235 00:13:58,320 --> 00:14:00,960 context of Carrie? Well, first I'll say I don't 236 00:14:00,960 --> 00:14:03,120 think it was intentional. I think that is that just comes 237 00:14:03,120 --> 00:14:06,320 naturally to some people. I certainly don't have my own 238 00:14:06,360 --> 00:14:10,640 version of that, but yeah, so does. 239 00:14:10,640 --> 00:14:12,360 Yeah. What does it mean to to study 240 00:14:12,360 --> 00:14:14,720 decision making? It's actually kind of hard to 241 00:14:14,720 --> 00:14:18,280 describe Kerry's research since it spans so many different 242 00:14:18,400 --> 00:14:21,480 categories. If you sort of take a look at 243 00:14:21,480 --> 00:14:25,000 his early work on affective forecasting with Dan Gilbert and 244 00:14:25,000 --> 00:14:29,000 Tim Wilson, some very amusing research on consumption 245 00:14:29,000 --> 00:14:31,840 behavior. And and then, you know, he's 246 00:14:31,840 --> 00:14:35,640 really, he's also covered all of what we consider the the 247 00:14:36,080 --> 00:14:38,320 behavioral economics classic, right? 248 00:14:38,320 --> 00:14:42,000 Like so mental accounting, anchoring, confirmation bias. 249 00:14:42,240 --> 00:14:44,720 He's looked at the endowment effect, actually teasing it 250 00:14:44,720 --> 00:14:47,800 apart from loss aversion, because some people kind of 251 00:14:47,800 --> 00:14:50,520 treat these as as two sides of the same coin. 252 00:14:50,520 --> 00:14:52,480 But Kerry will tell you they are. 253 00:14:52,560 --> 00:14:56,720 They're not. And today, very excitingly, he's 254 00:14:56,720 --> 00:14:58,880 one of the most prominent researchers looking at how 255 00:14:58,880 --> 00:15:03,240 humans interact with AI systems. Which, of course, is what this 256 00:15:03,240 --> 00:15:07,160 episode is all about. Yes, I'm giving you a quick 257 00:15:07,160 --> 00:15:10,440 preview of this episode. We look at AI systems from 258 00:15:10,440 --> 00:15:14,560 perspective of Kerr's work and especially kind of understanding 259 00:15:14,560 --> 00:15:18,040 this idea around the complexity of human behavior. 260 00:15:18,400 --> 00:15:21,600 And that if we don't understand that, this is to fall into these 261 00:15:21,600 --> 00:15:26,400 three common pitfalls that can lead to misalignment within AI 262 00:15:26,720 --> 00:15:30,920 recommendation systems and honestly so much more. 263 00:15:30,920 --> 00:15:33,640 In this episode, there's so much that we cover, including 264 00:15:33,640 --> 00:15:37,800 strategy and carry to our quick fire round of the season. 265 00:15:37,840 --> 00:15:42,400 To AI or not to AI and asking for his most controversial 266 00:15:42,400 --> 00:15:55,080 opinion about AI. Oh wow. 267 00:15:56,440 --> 00:15:59,720 Carrie, welcome. It's wonderful to have you on 268 00:15:59,720 --> 00:16:02,200 the show. Thank you for having me. 269 00:16:02,320 --> 00:16:05,040 Such a pleasure. It's been like a lifetime since 270 00:16:05,040 --> 00:16:07,160 we've hung out. It does feel that way. 271 00:16:08,480 --> 00:16:12,440 Our our beers in in Pittsburgh have gotten warm by now. 272 00:16:12,840 --> 00:16:14,800 At least our children's life. That's. 273 00:16:15,360 --> 00:16:16,440 Right. And for me, I think it's been a 274 00:16:16,440 --> 00:16:18,600 lifetime in terms of we haven't had a chance to meet before. 275 00:16:18,600 --> 00:16:23,240 So very excited to to shout. Me too, thanks for having me. 276 00:16:24,360 --> 00:16:28,000 So, Carrie, this season we're really focusing on AI and what 277 00:16:28,000 --> 00:16:30,160 behavioral science can say about AI. 278 00:16:30,240 --> 00:16:33,480 And you of course, have done some very exciting work at how 279 00:16:33,480 --> 00:16:37,400 people think about AI systems, how we perceive them, when we 280 00:16:37,400 --> 00:16:40,360 trust them and when we don't trust them, but also how these 281 00:16:40,360 --> 00:16:43,520 systems that are supposed to be these ultimate prediction 282 00:16:43,520 --> 00:16:46,600 machines, right? They can actually get it very 283 00:16:46,600 --> 00:16:48,920 wrong sometimes. And, and perhaps that could be 284 00:16:48,920 --> 00:16:52,360 because they're solving for the wrong goals or, or they don't 285 00:16:52,360 --> 00:16:56,600 have the, this, this understanding of human behavior 286 00:16:56,600 --> 00:17:00,080 built into them, or, or at least the, the machine learning 287 00:17:00,080 --> 00:17:02,840 engineers who are devising them have not thought about how 288 00:17:02,960 --> 00:17:06,680 humans think and behave. And, and so one of the points 289 00:17:06,680 --> 00:17:10,079 that comes through in your research is that when, when 290 00:17:10,079 --> 00:17:13,200 these AI systems base their recommendations, So in a 291 00:17:13,200 --> 00:17:15,960 recommender system, for example, when they base those 292 00:17:15,960 --> 00:17:18,920 recommendations on users digital footprints. 293 00:17:18,920 --> 00:17:23,560 So what was viewed or clicked or bought all all of those 294 00:17:23,560 --> 00:17:28,200 behaviors that are tangible and and seen captured by the system. 295 00:17:29,640 --> 00:17:33,680 When the systems are, are designed in this way, they they 296 00:17:33,680 --> 00:17:37,640 ignore what's arguably the most human thing about humans, right? 297 00:17:37,640 --> 00:17:41,360 And that's the gap between our intentions and our actions. 298 00:17:41,360 --> 00:17:45,840 So the the behaviors that we exhibit may not necessarily map 299 00:17:45,840 --> 00:17:49,320 to the things that we actually want to do, the ideal humans 300 00:17:49,320 --> 00:17:53,120 that we see ourselves as. So what I'm hoping you can dive 301 00:17:53,120 --> 00:17:56,400 into now is, is like what are algorithms doing wrong when 302 00:17:56,400 --> 00:17:59,760 they're basing their recommendations on users digital 303 00:17:59,760 --> 00:18:01,800 footprints? Right. 304 00:18:01,800 --> 00:18:04,520 So that's it. It's an amazing lead in and 305 00:18:04,520 --> 00:18:07,920 maybe it's useful to begin by thinking about like what they're 306 00:18:07,920 --> 00:18:10,000 trying to solve and how they do that. 307 00:18:11,120 --> 00:18:14,040 Let's begin by thinking about it as a spreadsheet, right? 308 00:18:14,840 --> 00:18:19,000 So an algorithm, you're telling it here are some columns to 309 00:18:19,000 --> 00:18:21,400 consider in the spreadsheet, right? 310 00:18:21,920 --> 00:18:25,120 I want you to use the information in those columns to 311 00:18:25,120 --> 00:18:29,480 make some kind of prediction. And like if you're thinking 312 00:18:29,480 --> 00:18:32,400 about it as regression, like those are your like X variables 313 00:18:32,400 --> 00:18:34,760 and the prediction is AY or outcome, right, that you're 314 00:18:34,760 --> 00:18:37,120 trying to predict. And then the data is really like 315 00:18:37,120 --> 00:18:40,040 the rows, right? So the users are the rows right 316 00:18:40,040 --> 00:18:46,200 in that spreadsheet. And so as algorithm designers, 317 00:18:46,320 --> 00:18:51,640 right, you're trying to think about, OK, well, what columns 318 00:18:51,640 --> 00:18:56,440 should I include or exclude in these kinds of predictions? 319 00:18:56,800 --> 00:18:59,560 Do I want to just like take all the data I have and like try to 320 00:18:59,560 --> 00:19:01,560 model something and see what it is? 321 00:19:01,920 --> 00:19:04,320 Do I think that there's like a certain number of columns that 322 00:19:04,320 --> 00:19:06,840 does something and then I'm trying to predict that? 323 00:19:07,400 --> 00:19:09,080 And then you're also trying to think about like, what is the 324 00:19:09,080 --> 00:19:10,840 thing that you're trying to predict and what's the best 325 00:19:10,840 --> 00:19:15,960 representation of that, right. So if I want to think about, for 326 00:19:15,960 --> 00:19:21,800 example, let's say that you are Netflix, right? 327 00:19:21,800 --> 00:19:27,800 And you want to show Samuel movies that he will enjoy, 328 00:19:28,040 --> 00:19:30,400 right? Like the outcome you're trying 329 00:19:30,400 --> 00:19:33,280 to predict is Samuel's enjoyment of the movies you want to 330 00:19:33,280 --> 00:19:34,840 recommend movies that he'll enjoy. 331 00:19:34,840 --> 00:19:38,400 So he stays on the platform, spends more time there, keeps 332 00:19:38,400 --> 00:19:43,560 his subscription. What kinds of features of films 333 00:19:43,560 --> 00:19:48,240 do you need to include in your prediction to like get a good 334 00:19:48,240 --> 00:19:50,600 prediction about movies that Samuel will like? 335 00:19:51,120 --> 00:19:56,640 Is it about like the actors, the genre, the length, right, like 336 00:19:56,640 --> 00:20:00,240 the visual representation, the language? 337 00:20:00,880 --> 00:20:03,800 So there are lots of different kinds of features, the director, 338 00:20:03,800 --> 00:20:04,800 right? Like there's lots of different 339 00:20:04,800 --> 00:20:08,200 kinds of features that you can include in that kind of 340 00:20:08,200 --> 00:20:10,880 prediction, right? And then the. 341 00:20:12,240 --> 00:20:15,760 Then you're trying to think about like, OK, well, what does 342 00:20:15,760 --> 00:20:17,840 it mean for Samuel to like the movie? 343 00:20:18,760 --> 00:20:20,560 So is it that he finishes the movie? 344 00:20:21,680 --> 00:20:24,280 Is that he finishes and rates the movie positively? 345 00:20:24,920 --> 00:20:26,840 Like, you get that like or love, right? 346 00:20:26,840 --> 00:20:29,520 Yeah. Is that like, or should we just 347 00:20:29,520 --> 00:20:31,480 use like or love? Like even if he watches the 348 00:20:31,480 --> 00:20:34,680 movie and he doesn't like or love it, like then should we not 349 00:20:34,760 --> 00:20:37,240 say that he liked the movie? I mean, I've had many 350 00:20:37,240 --> 00:20:39,480 experiences on Netflix where I ended up watching a whole thing 351 00:20:39,480 --> 00:20:40,600 and I was like, that was terrible, that. 352 00:20:40,720 --> 00:20:42,400 Was a crappy movie. Yeah, exactly. 353 00:20:43,680 --> 00:20:47,200 So there's a lot of different kinds of assumptions that we're 354 00:20:47,200 --> 00:20:52,600 making when we tried to take some abstract concept like 355 00:20:53,040 --> 00:20:55,840 enjoyment of a movie. How much will Samuel enjoy the 356 00:20:55,840 --> 00:20:58,920 movie, right. The idea behind that is that 357 00:20:58,920 --> 00:21:01,360 we're using, we're observing his behavior. 358 00:21:02,000 --> 00:21:05,040 So how much of the film, which movies he chooses, which movies 359 00:21:05,040 --> 00:21:07,840 he watches, how much of the movie he watches, how he rates 360 00:21:07,840 --> 00:21:11,400 it, right? We're observing his behavior to 361 00:21:11,520 --> 00:21:14,600 infer how he feels about the movie. 362 00:21:14,880 --> 00:21:18,400 And Economist would call these two kinds of states that we're 363 00:21:18,400 --> 00:21:21,960 using revealed preferences. So what he's doing, what he 364 00:21:21,960 --> 00:21:25,800 reveals about his behavior to infer people's normative 365 00:21:25,800 --> 00:21:27,520 preferences, what they actually believe. 366 00:21:28,960 --> 00:21:30,880 So let's take an example from Amazon. 367 00:21:30,880 --> 00:21:35,200 So many of us use Amazon for like as current for most of our 368 00:21:35,200 --> 00:21:37,480 purchases, right? Yeah, so. 369 00:21:37,760 --> 00:21:39,520 It's not returned. Yeah. 370 00:21:40,160 --> 00:21:44,560 So let's say that most people don't rate the things they buy 371 00:21:44,560 --> 00:21:47,040 on Amazon, but they might. Let's say that you buy diapers 372 00:21:47,040 --> 00:21:50,160 on Amazon for your child there. We could say the purchase is 1 373 00:21:50,160 --> 00:21:53,160 signal of liking the diaper and then repeat purchase would be a 374 00:21:53,160 --> 00:21:55,160 stronger signal of liking that diaper. 375 00:21:55,240 --> 00:21:58,120 It's harder to do something with a lamp which you buy and either 376 00:21:58,120 --> 00:22:00,000 return or you don't. The return is a clear signal. 377 00:22:00,000 --> 00:22:02,480 I don't like this lamp. Buying the lamp and not 378 00:22:02,480 --> 00:22:05,760 returning it probably means that you like it, but we don't know 379 00:22:05,760 --> 00:22:07,520 how much you like it. Do you buy, I think, other 380 00:22:07,520 --> 00:22:10,480 things from that brand, or do you go back and look at other 381 00:22:10,480 --> 00:22:12,000 lamps after you buy it? Right. 382 00:22:12,680 --> 00:22:16,600 So it's a really difficult problem because even with big 383 00:22:16,600 --> 00:22:21,320 data, we don't have a direct conduit into people's mental 384 00:22:21,320 --> 00:22:23,280 states. There's a lot of missing data. 385 00:22:23,840 --> 00:22:25,720 Yeah, there's a lot of missing data there. 386 00:22:25,720 --> 00:22:29,320 And there's a lot of assumptions that we're making about like 387 00:22:29,320 --> 00:22:31,800 what people are doing and what it means. 388 00:22:32,240 --> 00:22:35,840 And so the what people are doing are the revealed preferences and 389 00:22:35,840 --> 00:22:40,560 the what it means. Like, do we actually enjoy 390 00:22:41,280 --> 00:22:44,840 watching, you know, Love Island, right? 391 00:22:45,000 --> 00:22:47,600 Is that something that I would enjoy watching? 392 00:22:47,600 --> 00:22:49,680 Yes. Is that something that I would 393 00:22:49,760 --> 00:22:51,760 be happy about watching tomorrow? 394 00:22:52,240 --> 00:22:53,760 Those are different questions, right? 395 00:22:54,680 --> 00:22:58,640 Oh, and do you see yourself as the kind of person who watches 396 00:22:58,640 --> 00:23:02,160 Love Island and then the, you know, sort of the, the enjoyment 397 00:23:02,160 --> 00:23:05,560 while you're watching it versus the regret you may feel later 398 00:23:05,560 --> 00:23:08,720 after, you know, however many seasons of binging it? 399 00:23:08,760 --> 00:23:10,320 Yeah. Are are we still talking about 400 00:23:10,320 --> 00:23:14,760 me as a as an example? So here's an example. 401 00:23:14,760 --> 00:23:16,960 And it's not always that people are trying to pursue pleasure, 402 00:23:16,960 --> 00:23:19,560 right? So reading the New York Times is 403 00:23:19,560 --> 00:23:21,560 like a factory of sadness, right? 404 00:23:22,720 --> 00:23:25,400 It makes me so sad to read the New York Times, but I feel 405 00:23:25,400 --> 00:23:27,360 obligated to do it as a citizen, right? 406 00:23:27,480 --> 00:23:31,600 But it's something that like, I feel afterward, I feel good that 407 00:23:31,600 --> 00:23:33,040 I know what's happening in the world. 408 00:23:34,240 --> 00:23:37,240 Although if you like looked at how much, like if you were to 409 00:23:37,240 --> 00:23:40,160 measure my affective state while I was reading New York Times, I 410 00:23:40,200 --> 00:23:41,920 feel very unhappy most of the time. 411 00:23:42,520 --> 00:23:46,640 You know, So like, there are different kinds of goals that 412 00:23:46,640 --> 00:23:48,720 people have. There's a really amazing data 413 00:23:48,720 --> 00:23:51,760 set from the Neely Center at USC and they look at like people's 414 00:23:51,760 --> 00:23:54,080 emotional states while they're engaging in different kinds of 415 00:23:54,080 --> 00:23:57,520 social media. And like YouTube is one of the 416 00:23:57,520 --> 00:23:59,440 most positive kinds of experiences. 417 00:23:59,440 --> 00:24:01,880 And you could think about it like it's not necessarily a 418 00:24:01,880 --> 00:24:04,880 platform's design, but like why people go to YouTube is to 419 00:24:04,880 --> 00:24:07,480 learn, right? Like you go to learn about 420 00:24:07,880 --> 00:24:11,320 strollers or cars or like how to fix your refrigerator, right? 421 00:24:11,320 --> 00:24:14,560 Like that's like you're, you're learning things and it's kind of 422 00:24:14,560 --> 00:24:18,400 interesting, right? Like Twitter and like next door 423 00:24:18,400 --> 00:24:21,120 are like the least positive platforms, but like there you're 424 00:24:21,120 --> 00:24:23,520 kind of going there for like things that will make you 425 00:24:23,520 --> 00:24:27,920 unhappy, Like, and so there's a lot of kinds of assumptions that 426 00:24:27,920 --> 00:24:30,440 we're making when we're looking at people's behavior about like 427 00:24:30,440 --> 00:24:33,120 what the purpose of what they're doing is like what are they 428 00:24:33,120 --> 00:24:36,280 trying to optimize on? And so is the platform 429 00:24:36,280 --> 00:24:39,600 optimizing on what the user wants to optimize on in that 430 00:24:39,600 --> 00:24:41,800 situation. So that's like a long winded 431 00:24:41,800 --> 00:24:45,160 version of like what the difference between like 432 00:24:45,240 --> 00:24:48,400 normative preferences, like what I'm trying to optimize on as a 433 00:24:48,400 --> 00:24:53,840 user and then reveal preferences the platform trying to infer or 434 00:24:53,840 --> 00:24:57,520 what am I trying to do? And how well does my behavior 435 00:24:57,520 --> 00:24:59,280 predict what they think I'm trying to do? 436 00:25:00,760 --> 00:25:03,240 And yeah, and, and there are different ways of getting at 437 00:25:03,240 --> 00:25:06,200 these different versions of people's preferences, right? 438 00:25:06,400 --> 00:25:09,440 Do you think there there's a way to have your cake and eat it 439 00:25:09,440 --> 00:25:11,840 too? And, and thinking about, you 440 00:25:11,840 --> 00:25:14,840 know, how if you just take the Netflix example, you have 441 00:25:14,840 --> 00:25:17,880 multiple shelves, right? And you can have your like, you 442 00:25:17,880 --> 00:25:21,960 know, classics in one, which maybe is the version of you that 443 00:25:21,960 --> 00:25:24,480 you want to be. And then you have your Love 444 00:25:24,480 --> 00:25:28,200 Island shelf of, you know, all the trashy reality shows. 445 00:25:28,720 --> 00:25:33,760 Do you think that these multiple shelves can coexist or that, you 446 00:25:33,760 --> 00:25:36,520 know, will they just be overtaken by the algorithm 447 00:25:36,520 --> 00:25:40,200 learning over time when you continue watching Love Island 448 00:25:40,200 --> 00:25:44,240 that the classics kind of get pushed lower and lower into your 449 00:25:44,240 --> 00:25:45,640 recommendations? Yeah. 450 00:25:45,640 --> 00:25:50,840 I mean, I think that's a huge problem because it's hard for 451 00:25:50,840 --> 00:25:55,160 the platforms to observe like the heterogeneity of our 452 00:25:55,160 --> 00:25:57,560 preferences. And some platforms do do that, 453 00:25:57,560 --> 00:26:00,640 like Netflix does try to cluster different kinds of preferences 454 00:26:00,640 --> 00:26:02,760 that we have and they're building up like. 455 00:26:03,120 --> 00:26:05,880 But a lot of that is called from like collaborative filtering, 456 00:26:05,880 --> 00:26:09,600 which is it's showing us things that people who like the 457 00:26:09,600 --> 00:26:13,480 particular movie also. Yeah, Someone Like You who 458 00:26:13,480 --> 00:26:15,520 watched similar things would all. 459 00:26:15,600 --> 00:26:17,600 Here's something else that they watched and enjoyed. 460 00:26:18,400 --> 00:26:21,480 I, I think the a big problem is that our preferences are much 461 00:26:21,480 --> 00:26:26,640 like a, a Senate right or a, a, a group of people like a, a 462 00:26:26,640 --> 00:26:28,840 collective right. We have different preferences 463 00:26:28,840 --> 00:26:33,120 and sometimes they conflict. And which preference is dominant 464 00:26:33,120 --> 00:26:36,000 in a particular situation often depends on the goal we're trying 465 00:26:36,000 --> 00:26:39,840 to pursue or where we are in some particular context, right? 466 00:26:40,280 --> 00:26:43,200 You could think about at a family level, we need different 467 00:26:43,200 --> 00:26:45,480 Netflix accounts to make reasonable predictions. 468 00:26:45,480 --> 00:26:48,360 I have a friend who shares an account with his children and 469 00:26:48,360 --> 00:26:50,480 his wife. And it's like there's no 470 00:26:50,480 --> 00:26:53,680 coherence to what it's because like it could include like Coco 471 00:26:53,680 --> 00:26:56,720 Melon and like, you know. Spotify has that problem as 472 00:26:56,760 --> 00:27:00,160 well. Yeah, some like, you know, 473 00:27:00,520 --> 00:27:04,720 Scorsese film and like you wouldn't suggest like, you know, 474 00:27:05,160 --> 00:27:08,400 Scorsese film to a child. So, but like in our in, in my 475 00:27:08,400 --> 00:27:09,880 family, we have 3 accounts, right? 476 00:27:09,880 --> 00:27:12,000 One for my wife, one for me and one for our children. 477 00:27:12,240 --> 00:27:14,240 There's more, more coherence there. 478 00:27:14,240 --> 00:27:16,240 They, they, there's still some heterogeneity too. 479 00:27:16,440 --> 00:27:18,640 Yeah, but. Even within individual 480 00:27:18,640 --> 00:27:20,840 preferences, there's a lot of variety, right? 481 00:27:20,840 --> 00:27:23,560 Yeah. And if I recall, there was a 482 00:27:23,560 --> 00:27:27,080 point maybe early on in the Netflix days where Netflix asked 483 00:27:27,160 --> 00:27:30,440 people what kind of films would you like to see? 484 00:27:30,440 --> 00:27:34,200 And so it really directly got at their stated preferences. 485 00:27:34,600 --> 00:27:38,080 But that went so poorly because people didn't actually want to 486 00:27:38,080 --> 00:27:40,480 watch the films that they said they wanted to watch. 487 00:27:40,920 --> 00:27:43,480 And so they, you know, they lost users and so on. 488 00:27:43,960 --> 00:27:49,120 So 1 is this issue right? So Katie Milkman and Todd Rogers 489 00:27:49,120 --> 00:27:52,600 and Max Bazerman have a lovely paper called, I think Highbrow 490 00:27:52,600 --> 00:27:57,600 Films Gather Dust. And the basic idea there is 491 00:27:57,600 --> 00:28:01,360 they're looking at what movies people put into their watch 492 00:28:01,360 --> 00:28:05,160 lists. And the canonical example is 493 00:28:05,160 --> 00:28:08,320 like, Schindler's List, right? Most people feel like obligated 494 00:28:08,320 --> 00:28:10,640 to watch Schindler's List or think they should want to watch 495 00:28:10,640 --> 00:28:13,120 Schindler's List, right? Or movies like that that are 496 00:28:13,200 --> 00:28:16,240 like historical documentaries that are kind of sad, right? 497 00:28:17,920 --> 00:28:22,080 And then what do people actually watch, right? 498 00:28:22,880 --> 00:28:26,000 I think like the top movie in Netflix was like Red, which is 499 00:28:26,000 --> 00:28:29,760 like an action movie, right? So, you know, our extraction is 500 00:28:29,760 --> 00:28:33,120 also very popular in Netflix. When we're thinking about our 501 00:28:33,120 --> 00:28:36,720 choices, like what do we think are the movies that we should be 502 00:28:36,720 --> 00:28:38,720 watching? We're putting those things in in 503 00:28:38,720 --> 00:28:40,320 our watch list. And they're probably like 504 00:28:40,320 --> 00:28:43,320 classic movies or movies that are emotionally difficult to 505 00:28:43,320 --> 00:28:46,680 watch, things that require attention, things that will make 506 00:28:46,680 --> 00:28:49,720 us feel like better people and that we would be be proud of the 507 00:28:49,720 --> 00:28:52,600 next day having watched or spent that time that way, right? 508 00:28:53,280 --> 00:28:56,040 But when we look at when people log on and like you're exhausted 509 00:28:56,040 --> 00:28:58,440 from work and you're trying to pick something, you don't want 510 00:28:58,440 --> 00:29:00,000 to do that, right? Like. 511 00:29:00,760 --> 00:29:03,200 Exactly. That's that's like a eating your 512 00:29:03,480 --> 00:29:05,920 like you've already eaten your vegetables for the day, like 513 00:29:05,920 --> 00:29:07,520 you're, you're ready for dessert, right? 514 00:29:07,520 --> 00:29:11,880 So, you know, so in, in those kinds of cases, then people are 515 00:29:11,880 --> 00:29:13,760 watching more lowbrow films, right? 516 00:29:13,920 --> 00:29:16,400 So they're watching like comedies and action films and 517 00:29:17,360 --> 00:29:20,160 whatever is going to be more like emotionally palatable to 518 00:29:20,160 --> 00:29:22,760 them. And so this, there's this really 519 00:29:22,760 --> 00:29:27,800 big problem about platforms. This isn't just a Netflix 520 00:29:27,800 --> 00:29:30,760 problem. Platforms more generally are 521 00:29:30,760 --> 00:29:35,320 observing people's choices while they're making the choice. 522 00:29:36,160 --> 00:29:37,880 And that sounds very reasonable, right? 523 00:29:38,200 --> 00:29:41,160 Like I want to see what people choose when they choose and I'm 524 00:29:41,160 --> 00:29:43,400 going to show them more things that they choose. 525 00:29:45,080 --> 00:29:50,320 The the problem there is that like a lot of platforms then end 526 00:29:50,320 --> 00:29:54,360 up recording the weakest point in our self-control conflicts. 527 00:29:56,120 --> 00:29:58,400 Instead of reminding me that I should be watching these other, 528 00:29:58,400 --> 00:30:00,480 I want to watch these other things or I should feel like I 529 00:30:00,480 --> 00:30:01,960 should be watching these other things. 530 00:30:02,440 --> 00:30:05,840 And it's not measuring like how I felt about the movie I watched 531 00:30:05,920 --> 00:30:07,920 last night. It's like, how did you enjoy 532 00:30:07,960 --> 00:30:11,640 this movie right now, right? And like, do you go back and 533 00:30:11,640 --> 00:30:15,080 watch that movie again, right? Or do you watch other movies 534 00:30:15,080 --> 00:30:17,000 like that? So, you know, if you keep 535 00:30:17,000 --> 00:30:20,400 feeding people, you know dessert, right? 536 00:30:20,440 --> 00:30:22,400 And you're like, people eat dessert. 537 00:30:22,720 --> 00:30:25,560 Like if I put dessert in front of my kids and they eat dessert, 538 00:30:25,680 --> 00:30:27,240 right? And then you're like, oh, my 539 00:30:27,240 --> 00:30:29,000 kids like dessert, I'll give them more dessert, right? 540 00:30:29,040 --> 00:30:32,360 So like you keep doing that, like they'll keep eating it. 541 00:30:32,360 --> 00:30:34,480 But like, that's not to mean that like they wouldn't eat an 542 00:30:34,480 --> 00:30:37,080 apple if you put it in front of them, right? 543 00:30:37,480 --> 00:30:39,080 Or like they don't just want to eat dessert. 544 00:30:39,080 --> 00:30:41,200 They might like eat some apples and some strawberries or 545 00:30:41,200 --> 00:30:44,160 whatever, right? So the platforms even in like 546 00:30:44,160 --> 00:30:47,160 say, like, let's go to like an example, like Twitter, right? 547 00:30:47,160 --> 00:30:50,640 Or you know, and there or like TikTok or something like that, 548 00:30:50,640 --> 00:30:51,800 right? So they're looking at what 549 00:30:51,800 --> 00:30:54,400 you're consuming then. And like in Twitter, like you 550 00:30:54,400 --> 00:30:59,760 might be interested in like a salacious partisan news story, 551 00:30:59,760 --> 00:31:03,880 like JD Vance being like attracted to furniture, right? 552 00:31:03,880 --> 00:31:07,840 Like that might be funny, but that's not like, and you might 553 00:31:07,840 --> 00:31:09,960 like, look at it while you're scrolling, right. 554 00:31:09,960 --> 00:31:11,600 But like, that's not like what you. 555 00:31:11,800 --> 00:31:14,400 If the platform is all that, then people are done. 556 00:31:14,400 --> 00:31:19,120 Oh, yeah. And so the yeah, like, so like 557 00:31:19,160 --> 00:31:21,720 the the real problem with these kinds of platforms. 558 00:31:21,720 --> 00:31:24,240 And I think you can see it illustrated with like, sort of 559 00:31:24,240 --> 00:31:27,280 like the problem that Twitter's algorithm has had since it was 560 00:31:27,280 --> 00:31:29,520 like, acquired and became acts, right? 561 00:31:30,040 --> 00:31:35,000 Is that like, if you get too much into the want space, your 562 00:31:35,000 --> 00:31:39,200 platform is basically becoming like, like a flight from 563 00:31:39,280 --> 00:31:41,160 Pornhub, Right? Yeah. 564 00:31:41,160 --> 00:31:45,160 And like, no, seriously. And like, it's like people, 565 00:31:45,760 --> 00:31:50,040 people will eventually exit the platform if they don't feel good 566 00:31:50,040 --> 00:31:53,600 about like the next day about having how they spent their 567 00:31:53,600 --> 00:31:56,160 time. They might spend a lot of time 568 00:31:56,160 --> 00:31:59,560 on your platform when they're there because it's like shocking 569 00:31:59,560 --> 00:32:03,560 and salacious or like it's like, it's like emotionally evocative. 570 00:32:03,560 --> 00:32:06,000 And maybe they, maybe it's like moral outrage that they're 571 00:32:06,000 --> 00:32:08,880 feeling right. Or maybe it's like pleasure, or 572 00:32:08,880 --> 00:32:12,480 like some kind of hedonistic experience, which is fine to 573 00:32:12,480 --> 00:32:14,600 have, but you need a balance. Yeah. 574 00:32:14,600 --> 00:32:18,080 And I think that that is really the most critical piece of the 575 00:32:18,080 --> 00:32:20,680 puzzle, right? Is this impact on long term 576 00:32:20,680 --> 00:32:24,960 outcomes, both for the person's well-being and how they, you 577 00:32:24,960 --> 00:32:28,520 know, the relationship they have with the the product, but also, 578 00:32:28,520 --> 00:32:31,960 you know, for the for the company that's trying to get 579 00:32:31,960 --> 00:32:34,240 users to stay. If they're only looking at these 580 00:32:34,440 --> 00:32:38,080 short term outcomes and behaviors of what people watch, 581 00:32:38,280 --> 00:32:40,640 then they're not understanding the fact that in the long term 582 00:32:40,640 --> 00:32:42,960 this is actually bad for business because people are not 583 00:32:42,960 --> 00:32:44,880 happy with their behaviors long term. 584 00:32:45,440 --> 00:32:49,200 Yeah, and that's like one solution is looking at like 585 00:32:49,200 --> 00:32:52,520 let's just track behavior more long term, right rather than 586 00:32:52,520 --> 00:32:55,200 like what are people clicking on like what's what's the immediate 587 00:32:55,200 --> 00:32:58,960 kind of response, right. So what how long is it till 588 00:32:58,960 --> 00:33:00,560 people come back to the platform? 589 00:33:01,080 --> 00:33:03,280 Yeah, right. What is their like frequency on 590 00:33:03,400 --> 00:33:08,280 on time look like over a month? Also like can you look at when 591 00:33:08,280 --> 00:33:14,160 people are making choices, but like in an interval of time 592 00:33:14,160 --> 00:33:18,040 before they consume something like the watch list, right, 593 00:33:19,400 --> 00:33:21,080 right. Is there a gap between what 594 00:33:21,080 --> 00:33:22,880 they're putting in that watch list and what they end up 595 00:33:22,880 --> 00:33:25,640 consuming? And if there is, then there may 596 00:33:25,640 --> 00:33:29,760 be ways to try to think about how do we reconfigure the 597 00:33:29,760 --> 00:33:33,680 recommendation system So it's reflecting both kinds of 598 00:33:33,680 --> 00:33:35,720 desires. Because if you make a platform 599 00:33:35,720 --> 00:33:38,840 all shoulds, people won't log into it because there's no 600 00:33:38,840 --> 00:33:41,960 demand. People will not want to do that. 601 00:33:42,360 --> 00:33:46,080 But if you make a platform all wants, then people don't feel 602 00:33:46,080 --> 00:33:49,320 good about the experiences that they're having later on and 603 00:33:49,320 --> 00:33:52,280 whether or not it's detrimental for their emotional or physical 604 00:33:52,280 --> 00:33:54,000 well-being. We can put that aside. 605 00:33:54,240 --> 00:33:56,680 People just may not feel like that represents their 606 00:33:56,680 --> 00:34:00,720 preferences. A broad problem too, like 607 00:34:00,720 --> 00:34:06,120 whether or not we're actually modeling wants or shoulds, is 608 00:34:06,120 --> 00:34:11,159 that a lot of the kinds of decisions that people are making 609 00:34:11,159 --> 00:34:16,920 on these platforms are like habitual or they're done very 610 00:34:16,920 --> 00:34:21,159 intuitively. People are not like deliberating 611 00:34:21,159 --> 00:34:23,920 a long time before they're making these kinds of choices, 612 00:34:24,320 --> 00:34:26,840 right? And so my decision to click on a 613 00:34:26,840 --> 00:34:30,800 news story is like, you know, does the picture and like, 614 00:34:30,960 --> 00:34:33,280 however many characters look interesting. 615 00:34:34,120 --> 00:34:36,639 Does it look interesting from that point, Right. 616 00:34:37,120 --> 00:34:40,760 I'm not like, who is the poster? How many likes are there? 617 00:34:40,760 --> 00:34:42,920 How many retweets, right? How many comments are there? 618 00:34:43,000 --> 00:34:45,480 People do that when they're thinking very carefully about 619 00:34:45,480 --> 00:34:49,159 like, is this true or not? Right, but that. 620 00:34:49,520 --> 00:34:52,800 Sometimes. And like, and also like that. 621 00:34:52,800 --> 00:34:55,880 So there's like one level of people are not like fully 622 00:34:55,880 --> 00:34:57,320 thinking about all of the information. 623 00:34:57,320 --> 00:34:59,240 They're making more of an intuitive judgment than a 624 00:34:59,240 --> 00:35:01,720 deliberative judgment. And so I get picking more up on 625 00:35:01,720 --> 00:35:04,880 like how people think quickly than how people think carefully. 626 00:35:06,320 --> 00:35:09,560 And the other sort of thing is that like, you know, platforms 627 00:35:09,560 --> 00:35:13,320 run a lot of AB tests, but they don't make like major structural 628 00:35:13,320 --> 00:35:16,480 differences in the tests that they present to users. 629 00:35:16,960 --> 00:35:20,720 So we don't know like how much context is there. 630 00:35:20,720 --> 00:35:24,440 So we're then like, what we end up with is we're inferring 631 00:35:24,440 --> 00:35:30,600 people's preferences from like limited attention, right, from 632 00:35:30,600 --> 00:35:36,160 habits, and then like within a particular decision context that 633 00:35:36,160 --> 00:35:39,680 could skew their behavior. So let me give you an example of 634 00:35:39,680 --> 00:35:41,720 like why each of these is a problem, right? 635 00:35:42,440 --> 00:35:46,120 Habitual behavior. If we just observed smokers and 636 00:35:46,120 --> 00:35:49,400 like saw how they were smoking, we would think that like, most 637 00:35:49,400 --> 00:35:53,040 people who are smoking enjoy smoking and like, don't want to 638 00:35:53,040 --> 00:35:57,440 quit, right? If you survey most smokers, most 639 00:35:57,440 --> 00:35:59,760 smokers say that they do want to quit, right? 640 00:36:00,360 --> 00:36:02,760 So like, if we're looking at some kind of behavior that's 641 00:36:02,760 --> 00:36:06,160 habitual, like most people, I, I would get gather spend. 642 00:36:06,280 --> 00:36:08,520 Like the data suggests that most people are spending about an 643 00:36:08,520 --> 00:36:10,440 hour on like a platform like Instagram. 644 00:36:10,720 --> 00:36:14,960 I've been in a where I saw someone from Meta show a slide 645 00:36:14,960 --> 00:36:18,200 where they went from like the average user in India went from 646 00:36:18,200 --> 00:36:20,760 like 20 minutes a day to like 40 minutes a day. 647 00:36:21,000 --> 00:36:22,880 And they were like excited about that, right? 648 00:36:23,200 --> 00:36:25,560 That's great for Meta, but they thought the user was having a 649 00:36:25,560 --> 00:36:30,600 great time on Facebook too. I don't think anyone consciously 650 00:36:30,600 --> 00:36:34,200 wants to spend an hour a day on Instagram or an hour a day on 651 00:36:34,200 --> 00:36:36,640 Facebook and think like that's a good use of their time. 652 00:36:37,080 --> 00:36:39,240 I think if you ask people like how much time would you like to 653 00:36:39,240 --> 00:36:41,600 spend on these platforms, they would be like something like 654 00:36:41,640 --> 00:36:44,440 maybe like half an hour Max. But most people would probably 655 00:36:44,440 --> 00:36:46,680 say like, I want to spend like 5 to 10 minutes, right? 656 00:36:47,560 --> 00:36:50,440 I definitely want to spend, I don't want to spend more time on 657 00:36:50,440 --> 00:36:52,760 this platform than I spend with my children, right? 658 00:36:53,360 --> 00:36:55,880 Or like more time on this platform than I spend, like 659 00:36:55,880 --> 00:36:58,160 eating with my family, right? Yeah. 660 00:36:58,960 --> 00:37:06,040 So I guess we might want to jump to another topic, but before 661 00:37:06,040 --> 00:37:09,400 then I give you a chance to summarize the, you know, when we 662 00:37:09,400 --> 00:37:12,760 talk now quite a lot about the various kind of aspects of 663 00:37:13,160 --> 00:37:15,600 recommendations engine and algorithms to support us to make 664 00:37:16,160 --> 00:37:18,000 some more better decisions in some ways. 665 00:37:18,280 --> 00:37:20,720 What are the big problems that you've covered so far? 666 00:37:21,080 --> 00:37:22,800 Yeah. So there's sort of three, right. 667 00:37:23,560 --> 00:37:26,280 We're looking at thinking fast, not thinking slow. 668 00:37:26,320 --> 00:37:32,040 And that would be inattention habits and, you know, context 669 00:37:32,040 --> 00:37:36,120 effects on preferences. Algorithms are observing wants, 670 00:37:36,120 --> 00:37:39,440 not shoulds, right? What we want now, right now and 671 00:37:39,440 --> 00:37:42,440 not what we think we should have chosen or should choose in the 672 00:37:42,440 --> 00:37:44,280 future. And then algorithms are 673 00:37:44,280 --> 00:37:48,120 prioritizing like popular options and the status quo, like 674 00:37:48,120 --> 00:37:51,680 what we have data on, what most people like, right? 675 00:37:52,000 --> 00:37:57,200 And those kinds of three features are not necessarily 676 00:37:57,200 --> 00:38:01,800 inherently bad, but like they may lead us to make inferences 677 00:38:01,800 --> 00:38:04,880 about what users, users normative preferences, what they 678 00:38:04,880 --> 00:38:08,360 think, what they actually want, right, That are inaccurate 679 00:38:08,440 --> 00:38:10,440 because we're looking at the revealed behavior. 680 00:38:10,440 --> 00:38:14,640 So if I just look at what you're choosing intuitively, I may pick 681 00:38:14,640 --> 00:38:17,680 up on habits not like, you know, I see that you smoke and think 682 00:38:17,680 --> 00:38:22,280 that you like smoking, right? If I look at what you want, I 683 00:38:22,280 --> 00:38:25,560 might think that you don't like more highbrow films. 684 00:38:25,560 --> 00:38:27,160 You really like just lowbrow films. 685 00:38:27,560 --> 00:38:30,800 If I look at what you choose from like a recommendation 686 00:38:30,800 --> 00:38:34,240 system, right, that tends to prioritize popular options, I 687 00:38:34,240 --> 00:38:37,600 might think that, you know, you like Taylor Swift more than 688 00:38:37,600 --> 00:38:39,880 this, like other indie artists, right? 689 00:38:40,280 --> 00:38:42,920 Then, you know, you mentioned looking at long term behavior. 690 00:38:42,920 --> 00:38:45,520 That's a big kind of thing. Another thing that we didn't 691 00:38:45,520 --> 00:38:50,080 talk about is there are users who are modeling the behaviors 692 00:38:50,080 --> 00:38:52,920 that you would like users to model on that platform. 693 00:38:53,920 --> 00:38:57,120 Like there are people on Twitter who are reading interesting 694 00:38:57,120 --> 00:39:01,880 articles that are true, right? Or carrying like comedians that 695 00:39:01,880 --> 00:39:04,080 are telling funny jokes that everyone likes, right? 696 00:39:04,600 --> 00:39:09,680 You can teach or train the platform on that subset of users 697 00:39:09,680 --> 00:39:13,000 who are behaving the way that you would like users to behave. 698 00:39:13,280 --> 00:39:17,360 The analogy is let's take like we were way MO and we're making 699 00:39:17,360 --> 00:39:20,800 self driving cars and the way that you train a self driving 700 00:39:20,800 --> 00:39:22,840 car is it observes how a human would drive. 701 00:39:23,120 --> 00:39:27,720 Now, let's say that you wanted to train driving cars and you 702 00:39:27,720 --> 00:39:30,400 train them in Boston. Boston is full of terrible 703 00:39:30,400 --> 00:39:32,160 drivers, like self included, right? 704 00:39:32,160 --> 00:39:34,760 Like we do all kinds of terrible things like you have to break 705 00:39:34,760 --> 00:39:38,360 the law to drive in Boston. So like it's just impossible 706 00:39:38,360 --> 00:39:43,200 otherwise. So should we train self driving 707 00:39:43,200 --> 00:39:48,080 cars on the population? No, we want self driving cars to 708 00:39:48,080 --> 00:39:51,600 be trained on people who'd have a zero accident history, right? 709 00:39:51,600 --> 00:39:54,160 Canadians. You know, yeah, like nice 710 00:39:54,280 --> 00:39:57,040 drivers. We want like more self driving 711 00:39:57,040 --> 00:40:00,240 cars to be better versions of ourselves, right? 712 00:40:00,680 --> 00:40:04,560 And so like we kind of like an idea like why don't we train 713 00:40:04,560 --> 00:40:07,520 platforms on those better versions of ourselves or the 714 00:40:07,520 --> 00:40:11,680 users that we hope the platform was best served right? 715 00:40:12,560 --> 00:40:16,520 So one maybe quick thing that I want to touch upon before we 716 00:40:16,520 --> 00:40:21,480 jump into our quick fire round of the season is something that 717 00:40:21,480 --> 00:40:25,000 you've explored quite a bit, which is somewhat related to 718 00:40:25,000 --> 00:40:28,520 what we covered already, which has to do with algorithm 719 00:40:28,520 --> 00:40:32,040 aversion. So it's interesting to see how 720 00:40:32,040 --> 00:40:35,680 people react to algorithms and and what levels people feel 721 00:40:36,080 --> 00:40:39,000 comfortable with that and not comfortable and why and so on. 722 00:40:39,440 --> 00:40:41,160 Sure. I mean, I think a lot of like, 723 00:40:41,200 --> 00:40:44,720 you know, for the most part, algorithms are doing an amazing 724 00:40:44,720 --> 00:40:47,680 job in much of our lives. Like we've been spending the 725 00:40:47,680 --> 00:40:50,880 last however long talking about some of the problems with these 726 00:40:50,880 --> 00:40:52,800 algorithms. But like on the whole, 727 00:40:53,240 --> 00:40:56,480 algorithms are like dramatically improving our lives in many 728 00:40:56,480 --> 00:41:01,040 dimensions, right? So for example, you know, being 729 00:41:01,040 --> 00:41:06,240 able to, I can find the best route through Boston traffic at 730 00:41:06,240 --> 00:41:10,280 any time of day or night, right? And I know all the routes from 731 00:41:10,280 --> 00:41:14,640 like my office to my child's, like school or from my office to 732 00:41:14,640 --> 00:41:17,080 my home. But like an algorithm can guide 733 00:41:17,080 --> 00:41:20,200 me through the optimal route at any time of day or night. 734 00:41:20,200 --> 00:41:22,320 Like that's amazing. It can shave like 10 minutes off 735 00:41:22,320 --> 00:41:24,920 of my commute. That's, that's fantastic, right? 736 00:41:25,520 --> 00:41:29,440 Like, I think algorithms are also amazing at, like, solving 737 00:41:29,440 --> 00:41:33,680 like, the LA problem, which is when my cousins and my 738 00:41:33,680 --> 00:41:36,680 grandparents lived in Los Angeles and like, when I used to 739 00:41:36,680 --> 00:41:41,200 go there in college, like in the 1990s, which is, I shouldn't 740 00:41:41,200 --> 00:41:44,400 admit that. But like, like LA is a city 741 00:41:44,400 --> 00:41:47,880 where, like, the most amazing sushi place you could ever find 742 00:41:47,880 --> 00:41:51,280 is in, like, a strip mall on, like, in some weird place, 743 00:41:51,280 --> 00:41:52,760 right. Like, you need to know. 744 00:41:53,520 --> 00:41:56,360 You need to know that, like, that sushi place is in this 745 00:41:56,360 --> 00:42:00,080 place or you would drive by it and forever never see it. 746 00:42:00,080 --> 00:42:00,720 Never. Yeah. 747 00:42:00,960 --> 00:42:04,400 And like all the best like bars and clubs or like, you know, 748 00:42:04,400 --> 00:42:07,000 different kinds of social activities are all like, there's 749 00:42:07,000 --> 00:42:10,720 like, it's very hard to infer like what is good from just like 750 00:42:11,240 --> 00:42:13,960 casual observation, right? You need to have local knowledge 751 00:42:14,680 --> 00:42:16,880 and these algorithms. Now I can like go to LA and I 752 00:42:16,880 --> 00:42:21,560 just pull up Google Maps and I'm like, OK, find me find the best 753 00:42:21,640 --> 00:42:27,600 Mexican restaurant like in Westwood, right? 754 00:42:27,600 --> 00:42:30,080 Like or in North Hollywood or whatever. 755 00:42:30,080 --> 00:42:34,240 And it's gonna like, it would give me, like it gives me that 756 00:42:34,240 --> 00:42:36,200 note local knowledge immediately, right? 757 00:42:36,200 --> 00:42:38,640 So it's like aggregating all this information that like we 758 00:42:38,640 --> 00:42:42,120 didn't have before. So algorithms are amazing on all 759 00:42:42,120 --> 00:42:45,600 these kinds of dimensions. So, and we use algorithms too, 760 00:42:45,600 --> 00:42:49,920 Like, so you know, if Sandra, like if you came to Boston and 761 00:42:49,920 --> 00:42:55,640 Alien came to Boston and you were, you know, at a hotel and 762 00:42:55,680 --> 00:42:59,320 you were like, I'd like to go see Fenway Park because I'm 763 00:42:59,320 --> 00:43:01,800 really curious about where the Red Sox play, right? 764 00:43:02,360 --> 00:43:04,800 And you go downstairs and there's a hotel concierge and 765 00:43:04,800 --> 00:43:07,000 that person is an expert on Boston, right? 766 00:43:07,600 --> 00:43:10,440 They know where Fenway is. They like have maps, they can 767 00:43:10,440 --> 00:43:11,840 tell you how to get there, right? 768 00:43:12,440 --> 00:43:15,200 My guess is you walk right by that concierge and like check 769 00:43:15,200 --> 00:43:18,920 out Google Maps, right? So even if the concierge told 770 00:43:18,920 --> 00:43:20,960 you how to get there, you'd probably still open Google Maps, 771 00:43:21,000 --> 00:43:23,640 right? So we have in many kinds of 772 00:43:23,640 --> 00:43:25,840 cases, preferences for algorithms over people. 773 00:43:26,600 --> 00:43:28,800 Like the. So Google Maps example is the 774 00:43:28,800 --> 00:43:30,840 one that rings true all the time, right? 775 00:43:30,840 --> 00:43:34,920 I would rather use Google Maps than any human direction, right? 776 00:43:34,920 --> 00:43:37,360 Always. But there are also domains in 777 00:43:37,360 --> 00:43:41,120 which we don't want algorithms in lieu of people. 778 00:43:42,000 --> 00:43:48,040 For example, I would have a strong aversion to my son first 779 00:43:48,040 --> 00:43:51,560 romantic relationship be with an algorithm or a chat bot rather 780 00:43:51,560 --> 00:43:56,000 than a human being, right? I would feel despondent as a 781 00:43:56,000 --> 00:43:59,880 parent if like yes, his first love was a chat bot, right? 782 00:44:00,040 --> 00:44:02,240 Like that was feel terrible, right? 783 00:44:02,520 --> 00:44:07,440 And so that doesn't seem like a good substitute for like that 784 00:44:07,440 --> 00:44:12,720 human capacity, right? And I think a lot of people more 785 00:44:13,000 --> 00:44:16,400 immediately worry about algorithms like doing their 786 00:44:16,400 --> 00:44:19,520 work, right. And in some cases, like we don't 787 00:44:19,520 --> 00:44:24,080 necessarily like care. So people used to do statistics 788 00:44:24,080 --> 00:44:26,280 by hand. Now people use statistical 789 00:44:26,280 --> 00:44:28,640 packages, which are algorithms. They'll do their statistics. 790 00:44:28,640 --> 00:44:31,680 And I don't feel threatened as a professor that there's a 791 00:44:31,760 --> 00:44:33,360 computer program that does statistics. 792 00:44:33,360 --> 00:44:35,000 That's amazing. I don't have to do this part of 793 00:44:35,000 --> 00:44:37,400 the work, right? But those are the things that 794 00:44:37,400 --> 00:44:40,040 people necessarily don't identify with as their 795 00:44:40,040 --> 00:44:43,440 competency. And So what I think is when we 796 00:44:43,440 --> 00:44:48,680 really feel uncomfortable about algorithms doing something, it's 797 00:44:48,680 --> 00:44:52,120 some kind of competency that is important to who we are, our 798 00:44:52,120 --> 00:44:56,320 identity, right? And so I might not care about 799 00:44:56,360 --> 00:44:59,080 using a statistical package because for me, it's like the 800 00:44:59,080 --> 00:45:00,960 theory or the idea that's important. 801 00:45:01,280 --> 00:45:05,280 But for a statistician, they might be like, wait a minute, I 802 00:45:05,280 --> 00:45:08,360 don't want ChatGPT doing all of your statistics, right? 803 00:45:08,360 --> 00:45:11,480 Because I have important kinds of ideas or whatever, right? 804 00:45:11,480 --> 00:45:12,240 So, yeah. And. 805 00:45:12,880 --> 00:45:16,440 I wonder how much of that boils down to the fear of losing your 806 00:45:16,440 --> 00:45:18,720 job, right. You might be happy to use, you 807 00:45:19,120 --> 00:45:22,240 know, AI tools to accomplish your tasks as long as you're the 808 00:45:22,240 --> 00:45:26,760 primary one being paid and not losing your job. 809 00:45:27,000 --> 00:45:29,560 That's one of the biggest threats that people face, but 810 00:45:29,560 --> 00:45:34,160 they're also resistance to automation, which is fueled by 811 00:45:34,160 --> 00:45:37,200 algorithms that people have where it's more about things 812 00:45:37,200 --> 00:45:40,400 that they enjoy doing that they don't want to outsource. 813 00:45:40,880 --> 00:45:45,000 So Stefano Puentoni has this amazing paper where like, I 814 00:45:45,000 --> 00:45:48,240 don't enjoy baking bread, right? So if I had to make a lot of 815 00:45:48,240 --> 00:45:51,760 bread, I get a bread maker. There are a ton of people during 816 00:45:51,760 --> 00:45:54,640 the pandemic who really enjoyed making sourdough, right? 817 00:45:55,120 --> 00:45:57,760 Those people would not want to buy an automated bread maker. 818 00:45:57,760 --> 00:46:00,000 They want to knead the dough and bake it and observe what 819 00:46:00,000 --> 00:46:03,520 happens, right? I love cars. 820 00:46:03,720 --> 00:46:08,320 I have a manual transmission car cuz I enjoy having that kind of 821 00:46:08,320 --> 00:46:10,880 control right? To me an automatic is not as fun 822 00:46:10,880 --> 00:46:14,000 to drive cuz I have less control over it and the self driving car 823 00:46:14,000 --> 00:46:16,000 sounds terrible. Whereas there are many people 824 00:46:16,000 --> 00:46:19,760 who like driving is just a chore and they want an appliance that 825 00:46:19,760 --> 00:46:23,280 gets from one point to another and they don't really enjoy that 826 00:46:23,280 --> 00:46:26,440 kind of experience and so or having any control over it, they 827 00:46:26,440 --> 00:46:27,920 just don't want it to crash right? 828 00:46:28,200 --> 00:46:31,520 And so for them, an automated car would be delightful. 829 00:46:31,560 --> 00:46:34,680 And so in these kinds of spaces, it's really sort of how much do 830 00:46:34,680 --> 00:46:37,040 you identify with the thing that the algorithm would be doing? 831 00:46:37,640 --> 00:46:39,840 And is that a compliment to like what you're doing? 832 00:46:39,840 --> 00:46:43,280 Does it enable you to do something more easily and like 833 00:46:43,280 --> 00:46:45,680 more fluidly, like the statistics package? 834 00:46:46,120 --> 00:46:50,760 Or is it replacing like a core competency that you identify 835 00:46:50,760 --> 00:46:55,080 with, whether it's like making bread or like driving or writing 836 00:46:55,080 --> 00:46:58,920 screenplays or writing the news? Well, that seems like the 837 00:46:58,920 --> 00:47:01,440 perfect segue. Since our quick fire round is 838 00:47:01,520 --> 00:47:04,400 all about good or bad uses of AI. 839 00:47:04,760 --> 00:47:10,880 We are now going to share a a number of activities and ask 840 00:47:10,880 --> 00:47:15,320 you, Carrie, to tell us to AI or not to AI. 841 00:47:15,600 --> 00:47:18,160 Well, do it. Predict the outcome of a 842 00:47:18,160 --> 00:47:24,240 presidential election. Should but not there yet cuz 843 00:47:24,240 --> 00:47:26,680 there's so much change in historical context and there's 844 00:47:26,680 --> 00:47:29,640 not enough data that I I think that's an area where AI doesn't 845 00:47:29,640 --> 00:47:32,240 do well now but sure in the future why not? 846 00:47:34,520 --> 00:47:37,720 How about predict outcome on the football game? 847 00:47:37,720 --> 00:47:44,400 To AI or not to AI? Totally, yes, definitely not. 848 00:47:44,400 --> 00:47:47,280 Early season maybe, but later in the season it's gonna get more 849 00:47:47,280 --> 00:47:51,320 accurate. That's right, organize a summer 850 00:47:51,320 --> 00:47:58,520 internship program. What kind of summer internship 851 00:47:58,520 --> 00:48:00,920 program? Behavioral decision research, 852 00:48:00,920 --> 00:48:02,240 for example. Sure. 853 00:48:02,240 --> 00:48:04,800 I mean, if there is, if there are lots of good models to 854 00:48:04,800 --> 00:48:09,440 emulate, why not? What about the sign A better 855 00:48:09,440 --> 00:48:12,280 motorcycle? Sure, this is my favorite. 856 00:48:12,600 --> 00:48:15,920 Devise a diet plan using imagined satiety. 857 00:48:19,640 --> 00:48:22,080 Why not? Why not? 858 00:48:23,760 --> 00:48:27,640 This is when you came out with this paper, Thought for Food, 859 00:48:27,680 --> 00:48:31,200 where you show that people can habituate through imagining 860 00:48:31,200 --> 00:48:35,320 eating foods and desire those foods less often. 861 00:48:35,320 --> 00:48:38,520 I used to always think in my head about how I could create my 862 00:48:38,520 --> 00:48:42,480 own diet of just imagining that I'm eating any time I was 863 00:48:42,520 --> 00:48:44,240 inclined to lose a couple pounds. 864 00:48:44,480 --> 00:48:47,480 Like no need. Like take your reminder, like 865 00:48:47,480 --> 00:48:49,920 think about eating this. Exactly. 866 00:48:49,920 --> 00:48:54,800 Are you happy? No, yeah, that that could work. 867 00:48:55,080 --> 00:48:57,720 I, I, I would be excited to see a test of that. 868 00:48:58,840 --> 00:49:02,720 Well the human version of me never came up with the actual 869 00:49:02,720 --> 00:49:05,840 diet plan, so I I guess we'll have to rely on AI to do it. 870 00:49:06,560 --> 00:49:09,040 Yeah, but I mean, AI is amazing for things like putting a meal 871 00:49:09,040 --> 00:49:13,720 plan together, you know, devising an exercise routine. 872 00:49:14,800 --> 00:49:18,480 You know, I think it all depends on like the quality of the data 873 00:49:18,480 --> 00:49:22,880 out there for it's great. If it. 874 00:49:23,120 --> 00:49:26,960 Saves us from like if it saves us from the blog post Intro to 875 00:49:26,960 --> 00:49:32,280 the Imaginary Diet plan or like every recipe now gives you like 876 00:49:32,280 --> 00:49:33,560 the history of bread. Nice. 877 00:49:33,560 --> 00:49:38,000 So I think you mentioned maybe it was when we started the the 878 00:49:38,000 --> 00:49:40,960 podcast that you have two children and so this is a 879 00:49:40,960 --> 00:49:47,000 relevant one for, for you read a bedtime story to a child to AI 880 00:49:47,000 --> 00:49:51,360 or not AI. As a parent I feel I should do 881 00:49:51,360 --> 00:49:55,640 that, but there are times when I can't so I would say sure. 882 00:49:56,640 --> 00:49:57,920 Better than nothing. Yeah. 883 00:49:59,240 --> 00:50:01,360 All right. Act as human participants in 884 00:50:01,360 --> 00:50:05,880 experiments. I don't think we're there yet. 885 00:50:07,840 --> 00:50:12,080 But you think we will be. I think, you know, we do 886 00:50:12,080 --> 00:50:14,640 simulations like, there's lots of like game theoretic 887 00:50:14,640 --> 00:50:18,160 simulations. I think it can get there, and 888 00:50:18,160 --> 00:50:24,840 it's probably useful for like highlighting a concept, But you 889 00:50:24,840 --> 00:50:27,880 know, there's a lot of work suggesting that it's just 890 00:50:27,880 --> 00:50:32,960 recovering data that's out there so you can get basic kinds of 891 00:50:32,960 --> 00:50:36,720 people's response or demand to like price curves and things 892 00:50:36,720 --> 00:50:41,440 like that, that resemble what humans would do. 893 00:50:41,440 --> 00:50:43,760 But I think a lot of that's like scraping prices. 894 00:50:45,080 --> 00:50:48,640 So I think we'll get there and it can be a reasonable test run 895 00:50:48,840 --> 00:50:50,840 of like let's say you want to run a big field study. 896 00:50:51,000 --> 00:50:53,480 You can do a simulation with your program, but I don't think 897 00:50:53,480 --> 00:50:59,640 we're there yet. OK, final one, diagnose skin 898 00:50:59,640 --> 00:51:03,880 cancer. Yes, it's not as good as a 899 00:51:03,880 --> 00:51:06,960 dermatologist yet, but I think it'll get there. 900 00:51:08,480 --> 00:51:11,120 Maybe AI plus dermatologist is the right combo. 901 00:51:11,400 --> 00:51:14,600 I think that would be better than just a dermatologist. 902 00:51:15,360 --> 00:51:19,240 Yeah. And that brings us to maybe the 903 00:51:19,240 --> 00:51:22,360 final question, which kind of maybe indicates as well kind of 904 00:51:22,360 --> 00:51:26,200 your stance on AI. What is your most controversial 905 00:51:26,200 --> 00:51:30,480 opinion about AII? Think my biggest concern with 906 00:51:30,480 --> 00:51:31,200 AI. I'm not sure I have a 907 00:51:31,200 --> 00:51:33,720 controversial opinion about AI, but I would say my biggest 908 00:51:33,720 --> 00:51:38,680 concern is like it replacing human. 909 00:51:38,680 --> 00:51:42,920 Like large scale replacement of human interaction. 910 00:51:45,720 --> 00:51:52,120 Just people at home having relationships with a IS. 911 00:51:52,600 --> 00:51:55,120 And not that there's anything wrong with that from a 912 00:51:55,160 --> 00:51:58,280 philosophical or logical perspective, but it seems like a 913 00:51:58,280 --> 00:52:04,680 way to escape fundamental aspects of the human experience. 914 00:52:05,320 --> 00:52:07,520 You're really concerned about your son's the first 915 00:52:07,520 --> 00:52:09,800 relationship? No, I mean in general, like it's 916 00:52:09,800 --> 00:52:13,480 like a is are going to put up with our personality quirks and 917 00:52:13,480 --> 00:52:18,200 like grumpy moods, like more open, they're gonna be able to 918 00:52:18,200 --> 00:52:22,120 tailor a conversation to what we would find exciting. 919 00:52:22,160 --> 00:52:26,120 And like there's part of the human experience that is being 920 00:52:26,120 --> 00:52:29,120 validated and like feeling like your emotions and ideas and 921 00:52:29,120 --> 00:52:31,640 opinions are valid. And that's an important aspect. 922 00:52:31,640 --> 00:52:33,160 And I think that they will play to that. 923 00:52:33,760 --> 00:52:37,560 But I'm not sure that maybe they'll get there. 924 00:52:37,560 --> 00:52:41,720 But like I'm I'm worried that they will also include the kinds 925 00:52:41,720 --> 00:52:45,480 of challenging conversations or differences of ideas or opinions 926 00:52:46,320 --> 00:52:50,000 that interacting with other humans you know involves. 927 00:52:50,040 --> 00:52:51,640 Are you familiar with the company replica? 928 00:52:52,280 --> 00:52:56,120 Yes. I think one of the interesting 929 00:52:56,120 --> 00:52:59,080 subreddits out there that I randomly came into it at some 930 00:52:59,080 --> 00:53:02,920 point because I was interested with Replica being kind of a 931 00:53:02,920 --> 00:53:06,600 little bit of like before the very recent AI hype, they were 932 00:53:06,600 --> 00:53:10,240 already somewhat established as some form of AI friend platform. 933 00:53:10,800 --> 00:53:15,440 And they have a pretty big Replica subreddit, for example. 934 00:53:15,800 --> 00:53:21,760 And he's reading about people's kind of experiences from losing 935 00:53:21,760 --> 00:53:25,520 their account or talking about celebrating anniversaries and 936 00:53:25,520 --> 00:53:29,120 all of these things with these. I guess replicas has created 937 00:53:29,160 --> 00:53:30,960 mixed feelings. So yeah, I'm just curious if you 938 00:53:30,960 --> 00:53:33,040 had any thoughts. Yeah, I mean, I think that that 939 00:53:34,880 --> 00:53:40,400 there's a use for those kinds of proxies or parasocial 940 00:53:40,400 --> 00:53:44,280 relationships. One could be teaching people how 941 00:53:44,280 --> 00:53:47,160 to have different kinds of conversations. 942 00:53:47,160 --> 00:53:49,360 Like let's say you want to have a conversation with your child's 943 00:53:49,360 --> 00:53:52,800 teacher about an issue, right? Or your family member's sick. 944 00:53:53,320 --> 00:53:58,200 And like you want to think about how can I convey how I feel in a 945 00:53:58,200 --> 00:54:00,720 way that's sympathetic and empathetic to them? 946 00:54:00,720 --> 00:54:03,680 Or maybe you and your spouse are trying to negotiate what to do 947 00:54:03,680 --> 00:54:06,440 in the backyard, like what kind of landscaping to do, right? 948 00:54:06,760 --> 00:54:11,360 You could think about these kinds of platforms as mediation 949 00:54:11,360 --> 00:54:14,240 tools or teaching tools or simulation tools. 950 00:54:14,720 --> 00:54:18,080 And so for example, as department chair, I have to send 951 00:54:18,080 --> 00:54:20,000 a ton of different kinds of emails, right? 952 00:54:20,480 --> 00:54:25,000 And if I have an e-mail that I think could be particularly 953 00:54:25,000 --> 00:54:28,720 important to a variety of people and needs to be particularly 954 00:54:28,720 --> 00:54:32,800 sensitive in the way that I'm expressing information, I'll run 955 00:54:32,800 --> 00:54:37,040 it through like ChatGPT or Google Gemini or a Claude just 956 00:54:37,040 --> 00:54:43,160 to see, you know, is there a more positive or optimal tone 957 00:54:43,160 --> 00:54:45,680 for this e-mail or can I improve it? 958 00:54:45,680 --> 00:54:48,080 Right. So I think just as we can do 959 00:54:48,080 --> 00:54:51,360 that kind of simulation run, we're copy editing on the things 960 00:54:51,360 --> 00:54:58,280 that we're doing, having that kind of chance to simulate new 961 00:54:58,280 --> 00:55:01,520 or challenging or particularly important kinds of experiences 962 00:55:01,520 --> 00:55:05,120 and get training through that kind of simulation, I think 963 00:55:05,120 --> 00:55:08,200 would be amazing. What I worry about is that like 964 00:55:08,200 --> 00:55:12,520 we will end up substituting the simulation for reality. 965 00:55:13,040 --> 00:55:18,680 And maybe that's a, you know, bloodite kind of opinion, but 966 00:55:18,680 --> 00:55:23,800 like, that's that to me, like feels like what I would not want 967 00:55:23,800 --> 00:55:27,560 to do personally, Yeah. Yeah, and I think it's a it's a 968 00:55:28,120 --> 00:55:31,680 the memorable thing I remember from just kind of going on the 969 00:55:31,680 --> 00:55:34,760 replica subreddit was just someone one of the most liked 970 00:55:34,760 --> 00:55:39,960 posts or upvoted was someone that had basically found a way 971 00:55:39,960 --> 00:55:44,240 to hack the algorithm to be a good friend, which in this 972 00:55:44,240 --> 00:55:47,080 community meant basically like downvoting times. 973 00:55:47,080 --> 00:55:52,320 The the AI friend would say things that are like critical or 974 00:55:53,720 --> 00:55:58,280 not positive and uploading everything basically that when 975 00:55:58,280 --> 00:56:00,880 it said things that were friendly so that it would ensure 976 00:56:00,880 --> 00:56:03,400 that it had a friend, virtual friend in this case, that was 977 00:56:03,400 --> 00:56:05,880 always supportive, that was always super, super positive and 978 00:56:05,880 --> 00:56:07,920 so on. And it kind of becomes like a 979 00:56:07,920 --> 00:56:11,280 little bit of dystopian in terms of what you were describing here 980 00:56:11,280 --> 00:56:14,520 in terms of a bit Huxley. And it becomes very unfortunate, 981 00:56:14,720 --> 00:56:16,920 but also to tie things up even more. 982 00:56:16,920 --> 00:56:20,080 So a need to have your kind of work when it really comes to 983 00:56:20,080 --> 00:56:23,040 understanding how this interchange happens between 984 00:56:23,040 --> 00:56:25,680 algorithms, how they're built, how they're recommending things 985 00:56:25,680 --> 00:56:28,520 for us, but also how we're kind of using them in turn. 986 00:56:28,880 --> 00:56:31,720 And understanding that kind of interchange is, is so important 987 00:56:31,720 --> 00:56:32,920 right now. And I'm really happy that we get 988 00:56:32,920 --> 00:56:34,640 a chance to explore some of it today. 989 00:56:34,640 --> 00:56:38,280 I feel like we could talk much more about much of this, but 990 00:56:38,320 --> 00:56:41,440 thanks so much for joining and taking the time and for all of 991 00:56:41,440 --> 00:56:43,400 your great work on on this topic. 992 00:56:43,400 --> 00:56:45,520 And. Thank you for your interest in 993 00:56:45,520 --> 00:56:48,360 like, giving a platform to these kinds of ideas and 994 00:56:48,360 --> 00:56:50,960 conversations. It's great to think about these 995 00:56:50,960 --> 00:56:54,160 things together. And that's a wrap. 996 00:56:54,480 --> 00:56:57,160 You've been listening to the Behavioral design podcast 997 00:56:57,440 --> 00:57:00,000 brought to you by Habit Weekly and Nuanced Behavior. 998 00:57:00,360 --> 00:57:02,680 Sam and Eileen tell me. This season is packed with 999 00:57:02,680 --> 00:57:06,560 incredible insights about behavioral design and AI, so be 1000 00:57:06,560 --> 00:57:09,120 sure to subscribe and share the podcast with your friends, 1001 00:57:09,360 --> 00:57:11,520 though you might want to keep it away from your enemies. 1002 00:57:12,880 --> 00:57:15,480 In case you haven't noticed, I'm an AI voice. 1003 00:57:16,680 --> 00:57:19,440 Yep, pretty crazy. Quite the improvement since last 1004 00:57:19,440 --> 00:57:21,440 season's AI outro, don't you think? 1005 00:57:22,720 --> 00:57:25,360 If you'd like to collaborate with us at Nuance Behavior, 1006 00:57:25,560 --> 00:57:28,280 where we use behavioral design to craft digital products with 1007 00:57:28,280 --> 00:57:32,640 Nuance, e-mail us at hello@nuancebehavior.com or book 1008 00:57:32,640 --> 00:57:35,880 a call directly on our website, nuancebehavior.com. 1009 00:57:37,320 --> 00:57:40,720 A special thanks to the amazing Dave Pizarro for our show music, 1010 00:57:40,960 --> 00:57:43,840 and to Mei Chen Yap and April English for their help in 1011 00:57:43,840 --> 00:57:45,760 producing and publishing this episode. 1012 00:57:46,280 --> 00:57:49,040 Thanks again for tuning in. We'll be back soon with another 1013 00:57:49,040 --> 00:57:52,240 exciting conversation where behavioral design and AI 1014 00:57:52,240 --> 00:58:17,240 intersect. Oh. 1015 00:58:52,120 --> 00:58:56,200 And do you see yourself as the kind of person who watches Love 1016 00:58:56,200 --> 00:58:58,080 Island and enjoys watching Love Island? 1017 00:58:58,080 --> 00:59:01,360 And then the enjoyment while you're watching it versus the 1018 00:59:01,360 --> 00:59:05,080 regret you may feel later after, you know, however many seasons 1019 00:59:05,080 --> 00:59:06,600 of binging it? Yeah. 1020 00:59:06,920 --> 00:59:08,600 Are are we still talking about me as a as an example?