1 00:00:00,040 --> 00:00:07,320 Welcome to CXO Talk episode 860. I'm Michael Krigsman, and we are 2 00:00:07,320 --> 00:00:11,720 discussing AI and the future of work with Doctor Terry 3 00:00:11,720 --> 00:00:16,920 Signowski, a pioneer in computational neuroscience and 4 00:00:16,920 --> 00:00:20,800 deep learning. He's the Francis Crick Chair at 5 00:00:20,800 --> 00:00:25,040 the Salk Institute for Biological Studies and is also a 6 00:00:25,040 --> 00:00:29,720 distinguished professor in the Department of Neurobiology at UC 7 00:00:30,560 --> 00:00:35,680 San Diego. He is also Chairman of the Nur 8 00:00:35,680 --> 00:00:41,800 IPS Foundation, which runs the largest, most prestigious AI 9 00:00:41,800 --> 00:00:46,600 conference in the world. Terry has published over 500 10 00:00:46,600 --> 00:00:51,280 scientific papers and 12 books, including his most recent, 11 00:00:51,280 --> 00:00:57,240 ChatGPT and The Future of AI, and has won more awards than I 12 00:00:57,240 --> 00:00:59,640 can name. I have a background in physics, 13 00:00:59,880 --> 00:01:05,200 which is basically a license to work on anything, but my real 14 00:01:05,200 --> 00:01:08,520 focus is on the brain. I've won awards in neuroscience, 15 00:01:08,520 --> 00:01:10,640 the most recently the Brain prize, which is the highest 16 00:01:10,640 --> 00:01:14,600 award. And you know, I was a pioneer in 17 00:01:14,600 --> 00:01:17,640 the early days in the 80s developing learning algorithms 18 00:01:17,640 --> 00:01:19,720 for neural networks with Jeff Hinton and others. 19 00:01:20,160 --> 00:01:23,600 And so I have the, these two different backgrounds, which 20 00:01:23,600 --> 00:01:27,160 I've now have reached the point where this is really what's 21 00:01:27,160 --> 00:01:28,800 exciting is they're coming together. 22 00:01:29,680 --> 00:01:31,920 They're, they're actually synergistic. 23 00:01:32,320 --> 00:01:36,560 And So what we learned in the brain is going to help better AI 24 00:01:36,880 --> 00:01:40,080 in the future, and what we learned in AI is actually 25 00:01:40,080 --> 00:01:42,600 helping us in neuroscience. So this is really an exciting 26 00:01:42,600 --> 00:01:47,600 time to be doing research. How do you model AI on the 27 00:01:47,600 --> 00:01:49,560 brain? Maybe we should start there. 28 00:01:49,920 --> 00:01:53,960 Back in the 20th century, artificial intelligence was 29 00:01:53,960 --> 00:01:58,640 really focused on things that ran on puny digital computers, 30 00:01:58,640 --> 00:02:02,800 namely symbols, logic and rules. And there's only so far you can 31 00:02:02,800 --> 00:02:04,840 go with that, especially when you're dealing with a, a 32 00:02:04,840 --> 00:02:11,280 complex, very complex world with lots of uncertainties in it, 33 00:02:11,320 --> 00:02:12,800 right? And so you need, you need a 34 00:02:12,800 --> 00:02:17,160 much, much more computation and, and, and, and now we know, in 35 00:02:17,160 --> 00:02:22,120 fact, you know, that nature has solved all these problems, 36 00:02:22,360 --> 00:02:26,600 vision, you know, coordination, motor and so forth. 37 00:02:27,120 --> 00:02:29,840 And so we're learning now we popped the hood and now we're 38 00:02:29,840 --> 00:02:31,680 learning. What do you take from the brain? 39 00:02:31,920 --> 00:02:36,840 Massively parallel, highly interconnected parameters that 40 00:02:36,840 --> 00:02:40,320 called synapses in the brain and learning that you learn the 41 00:02:40,320 --> 00:02:42,360 strengths of those synapses with experience. 42 00:02:42,360 --> 00:02:46,760 And, and now it's big data and that has transformed all of AI 43 00:02:46,840 --> 00:02:49,360 And, but the point is though, we're dealing with the same 44 00:02:49,360 --> 00:02:51,720 mathematical structures, right? These neural networks are very 45 00:02:51,720 --> 00:02:53,880 simplified versions of brains, but they have the same 46 00:02:53,880 --> 00:02:58,400 mathematical way of of of analyzing and making progress. 47 00:02:58,840 --> 00:03:03,880 How does the brain organize information differently from 48 00:03:04,480 --> 00:03:08,280 machines and neural networks? And I just want to mention that 49 00:03:08,280 --> 00:03:12,400 the prominent data scientist Anthony Scrifignano has helped 50 00:03:12,400 --> 00:03:14,400 me prepare some of these questions. 51 00:03:14,800 --> 00:03:17,800 The general principles I've, I've given you take you so far, 52 00:03:17,800 --> 00:03:24,240 but actually nature has evolved of specialized circuits for 53 00:03:24,240 --> 00:03:28,360 specialized problems in, in different species in order for 54 00:03:28,360 --> 00:03:32,720 survival. And you know the, the different, 55 00:03:32,720 --> 00:03:34,120 there are many, many differences. 56 00:03:35,280 --> 00:03:40,560 There is the, the deep learning architecture that is used today 57 00:03:41,320 --> 00:03:44,000 was inspired by the cortex, the cerebral cortex, which is the 58 00:03:44,000 --> 00:03:48,400 highest level on top of the brain that humans use for, you 59 00:03:48,400 --> 00:03:52,280 know, the knowledge base. And there's a hundred other 60 00:03:52,280 --> 00:03:56,000 brain areas that are equally important for her being able to 61 00:03:56,000 --> 00:03:58,600 handle the complexities of the world and to survive. 62 00:03:58,960 --> 00:04:03,240 And, and a lot of those are ones that are right now are being 63 00:04:03,240 --> 00:04:06,360 integrated into the, the, the next generation of AI. 64 00:04:06,720 --> 00:04:08,960 And I'll just give you one example, reinforcement learning. 65 00:04:09,560 --> 00:04:12,560 So reinforcement learning is, is the part of the brain that is, 66 00:04:12,560 --> 00:04:16,680 is found in almost all species and has to do with being able 67 00:04:16,680 --> 00:04:22,079 to, first of all, from experience, predict whether a 68 00:04:22,079 --> 00:04:25,320 particular action is going to give you a reward or not. 69 00:04:25,720 --> 00:04:28,320 And you, you build that up over many, many trials. 70 00:04:28,320 --> 00:04:31,440 You know, if you're wrong, you twiddle awaits. 71 00:04:31,960 --> 00:04:34,440 And that's actually used for what's called procedural 72 00:04:34,440 --> 00:04:37,600 learning, learning how to play tennis, for example, or learning 73 00:04:37,600 --> 00:04:44,520 how to do us, you know, solve problems that are, you know, in 74 00:04:44,520 --> 00:04:48,160 law, in medicine, all of that is, is done. 75 00:04:48,280 --> 00:04:51,800 It becomes automatic. And, and, and now that's being 76 00:04:51,800 --> 00:04:55,400 incorporated into AI and it's, it's going to help, you know, 77 00:04:55,400 --> 00:04:58,240 advance it and, and make it more, much more broad and much 78 00:04:58,240 --> 00:05:01,920 more reliable. Can you drill into this a little 79 00:05:01,920 --> 00:05:05,240 bit? How does how do you make that 80 00:05:05,280 --> 00:05:11,520 transference of understanding the brain into encoding this 81 00:05:11,520 --> 00:05:14,800 into a digital system? It's all about architecture. 82 00:05:14,840 --> 00:05:16,600 That is to say, you have the same units, but you could put 83 00:05:16,600 --> 00:05:18,600 them together in different layers and with different 84 00:05:18,600 --> 00:05:21,160 connectivity. And you know, in Transformers, 85 00:05:21,160 --> 00:05:25,080 which is the, the, the, the heart of large language models, 86 00:05:25,640 --> 00:05:28,720 there's, there's special things that were added in order to be 87 00:05:28,720 --> 00:05:32,200 able to make it a, a, a powerful language learner. 88 00:05:33,400 --> 00:05:36,200 And by the way, that it was trained completely on the basis 89 00:05:36,200 --> 00:05:39,280 of just predicting the next word in a sentence, right? 90 00:05:39,280 --> 00:05:42,280 It, it wasn't given any special training on any special tasks. 91 00:05:42,280 --> 00:05:45,600 But now that we trained it up on a huge database that can solve 92 00:05:45,600 --> 00:05:48,960 many, many natural language problems that before were very 93 00:05:48,960 --> 00:05:51,520 difficult and then required specialized networks. 94 00:05:52,800 --> 00:05:56,600 And so that's one step closer to the fact that we know we have a 95 00:05:56,600 --> 00:05:59,800 very general, we, our brains are very general in terms of being 96 00:05:59,800 --> 00:06:01,520 able to handle lots of different problems. 97 00:06:02,160 --> 00:06:06,880 But OK, so reinforcement learning in particular is, is 98 00:06:06,880 --> 00:06:12,280 really, I, I think a part of the brain that is absolutely 99 00:06:12,280 --> 00:06:14,800 essential. It's all about practice. 100 00:06:14,880 --> 00:06:17,720 If you want to learn how to play tennis, there's two options. 101 00:06:17,720 --> 00:06:19,480 You can read books about tennis, right? 102 00:06:19,480 --> 00:06:22,240 That's the declarative part, the cortical part, the conscious 103 00:06:22,240 --> 00:06:24,160 part. Or you could go out on the 104 00:06:24,160 --> 00:06:27,000 tennis court and start hitting them and getting feedback and 105 00:06:27,000 --> 00:06:29,440 getting better and better day after day, week after week. 106 00:06:29,800 --> 00:06:34,680 And, and that eventually automatizes it so that you don't 107 00:06:34,680 --> 00:06:37,400 have to think about it anymore. And, and, but of course, that 108 00:06:37,400 --> 00:06:38,800 means you can't explain it to anybody. 109 00:06:40,480 --> 00:06:42,760 So if someone asks you, how did you do the serve? 110 00:06:42,960 --> 00:06:45,200 Well, you put your hand up and you hit the ball right. 111 00:06:45,200 --> 00:06:46,880 And that does it really help them? 112 00:06:47,320 --> 00:06:51,480 What about explain ability? Can you discuss the challenges 113 00:06:51,480 --> 00:06:57,720 of explainability that arise from neural networks and machine 114 00:06:57,720 --> 00:06:59,880 learning? And this has become very 115 00:06:59,880 --> 00:07:02,760 important and in business and and Justin. 116 00:07:03,160 --> 00:07:06,760 Let me first point something out that's obvious, which is that 117 00:07:07,000 --> 00:07:13,080 everybody has a brain and uses it to solve problems and doesn't 118 00:07:13,080 --> 00:07:17,200 know how it does that right. The brain is not explainable is 119 00:07:17,200 --> 00:07:20,040 that it doesn't give us insight to how we solve the problem, 120 00:07:20,040 --> 00:07:23,160 right? But, and we make mistakes, we 121 00:07:23,160 --> 00:07:25,760 hallucinate. Actually, we we make things up, 122 00:07:26,520 --> 00:07:29,200 filling in the blanks, right? We can't remember in detail what 123 00:07:29,200 --> 00:07:31,480 happened, you know, a couple years ago, but we can kind of 124 00:07:31,480 --> 00:07:33,920 fill it in. So, you know, this is not, you 125 00:07:33,920 --> 00:07:39,040 know, it's unusual that we're creating a system that is like 126 00:07:39,040 --> 00:07:42,040 the brain in some ways that it's also going to have some of the 127 00:07:42,040 --> 00:07:43,880 problems of the brain. So OK, but now let's look at 128 00:07:43,880 --> 00:07:46,440 explainability. Explainability is not just one 129 00:07:47,640 --> 00:07:50,560 level of understanding and, and let's take medicine, right? 130 00:07:51,280 --> 00:07:53,800 Well, you go to the doctor, you get a pill and the doctor will 131 00:07:53,800 --> 00:07:57,080 tell you, oh, it's going to help your immune system, you know, 132 00:07:57,080 --> 00:07:59,520 and maybe that's good enough for the patient, but it's actually 133 00:07:59,520 --> 00:08:03,320 not good enough if you're, you know, trying to do research on 134 00:08:03,320 --> 00:08:06,600 medicines and try to understand, you know, how it works. 135 00:08:07,520 --> 00:08:09,080 And that's another level explanation. 136 00:08:09,080 --> 00:08:13,320 But, you know, most, most people would be, you know, you know, 137 00:08:13,400 --> 00:08:15,600 don't have the background to understand that you have to 138 00:08:15,600 --> 00:08:19,960 have, you know, be an expert. But even that, OK, there's a lot 139 00:08:19,960 --> 00:08:23,280 of going on in the brain that that even experts don't 140 00:08:23,280 --> 00:08:25,600 understand. It's just really complicated, 141 00:08:26,560 --> 00:08:29,840 You know, all of the different molecules interacting and 142 00:08:29,840 --> 00:08:32,320 signals going back and forth. You know, we, we have a very 143 00:08:32,799 --> 00:08:37,480 crude understanding of, of really how, how nature put 144 00:08:37,480 --> 00:08:41,760 together the, all the different organs in the body and how they 145 00:08:41,760 --> 00:08:44,320 interact with each other. We're discovering that the, the 146 00:08:44,320 --> 00:08:48,560 gut actually has a connectivity, you know, and send signals back 147 00:08:48,560 --> 00:08:51,280 and forth with the brain, the gut, you know, the microbiome, 148 00:08:51,520 --> 00:08:53,480 right? This is something we didn't know 149 00:08:53,480 --> 00:08:56,320 until recently and we're learning things every day now. 150 00:08:56,800 --> 00:08:59,200 But you can go even down deeper. You can go down and 151 00:08:59,200 --> 00:09:01,840 understanding, you know, the actual details of the molecules 152 00:09:01,840 --> 00:09:04,440 and, and this is physics explanation, right? 153 00:09:04,440 --> 00:09:07,480 This is something that is at the level where you have equations 154 00:09:07,800 --> 00:09:09,600 and, and that level of explanation. 155 00:09:09,600 --> 00:09:12,360 All these explanations, you know, are, are, have different 156 00:09:13,280 --> 00:09:17,720 ways of, of giving us insights and, and you know, the, the one 157 00:09:17,720 --> 00:09:19,520 that we're looking for. Is it words? 158 00:09:19,520 --> 00:09:21,840 No, I think we're, we're really, we really want to get an 159 00:09:21,840 --> 00:09:24,160 understanding of what's happening at that deepest level. 160 00:09:24,440 --> 00:09:28,040 And that's beginning to happen because we have a lot of very, 161 00:09:28,040 --> 00:09:32,640 very good mathematicians and people, engineers with the 162 00:09:32,640 --> 00:09:35,160 training and machine learning are digging down into these 163 00:09:35,160 --> 00:09:40,520 networks and figuring out this the actual way that they are, 164 00:09:40,880 --> 00:09:43,200 you know, how the information flows through the network and 165 00:09:43,200 --> 00:09:46,680 how how it's able to do the amazing things that it does. 166 00:09:46,680 --> 00:09:50,040 And and, you know, this is really where we're going to get 167 00:09:50,040 --> 00:09:53,280 the real understanding, not just in terms of some explanation 168 00:09:53,280 --> 00:09:57,160 that, you know, you could live, the explanation that you might 169 00:09:57,160 --> 00:10:00,120 use just to make you think that you've understood something. 170 00:10:00,520 --> 00:10:07,400 As we rely more and more on AI to make decisions, including 171 00:10:07,800 --> 00:10:13,520 medical decisions, you, you just alluded to that this issue of 172 00:10:13,520 --> 00:10:17,640 explainability will become more and more important. 173 00:10:17,840 --> 00:10:23,240 So the real question for from that I have is can we ever 174 00:10:23,240 --> 00:10:27,520 really understand how the machine is making these 175 00:10:27,520 --> 00:10:30,080 decisions? And then what do we do about the 176 00:10:30,080 --> 00:10:35,040 fact that we that we're asking machines to ultimately decide 177 00:10:35,040 --> 00:10:40,000 important aspects of our lives and we don't know fully what's 178 00:10:40,000 --> 00:10:42,760 going on? You shouldn't think of AI as 179 00:10:42,760 --> 00:10:45,400 being a human. It's not right. 180 00:10:45,720 --> 00:10:47,360 It's it has special capabilities. 181 00:10:47,360 --> 00:10:50,440 It can, it can take in much more knowledge than any single human, 182 00:10:50,440 --> 00:10:52,440 right. So it's, it's really in some way 183 00:10:52,440 --> 00:10:55,880 already super intelligent, although people argue that it's 184 00:10:55,880 --> 00:10:58,320 not intelligent, but nonetheless it's helpful. 185 00:10:58,680 --> 00:11:01,120 So AI is a tool. It's going to help you solve 186 00:11:01,120 --> 00:11:05,400 problems and, and you talk about medical of decisions that have 187 00:11:05,400 --> 00:11:07,920 to be made. So a study was done. 188 00:11:07,920 --> 00:11:11,400 It was published in nature. Nature is the gold standard for 189 00:11:11,400 --> 00:11:15,280 science, all sciences and engineering. 190 00:11:16,000 --> 00:11:20,000 And, and what they did was a study on lesion skin lesions, 191 00:11:20,200 --> 00:11:23,040 some of which are cancerous and some of which are benign, right? 192 00:11:23,160 --> 00:11:25,640 And it's like 2000. And what they did was they 193 00:11:25,640 --> 00:11:31,080 compared experts or panel of experts with the best AI 194 00:11:31,080 --> 00:11:34,320 solution. AI was given thousands and 195 00:11:34,320 --> 00:11:38,760 thousands of examples, images of of lesions and it was taught to 196 00:11:38,760 --> 00:11:41,520 classify them. Now it turned out that both 197 00:11:41,520 --> 00:11:44,040 groups had a performance of about 90%. 198 00:11:44,040 --> 00:11:47,640 They were correct in telling you what kind of lesion or whether 199 00:11:47,640 --> 00:11:50,080 or not it was cancerous at about the same level. 200 00:11:50,360 --> 00:11:54,920 OK, so that's interesting. However, if they let the doctor 201 00:11:54,920 --> 00:11:58,640 use the AI in making the diagnosis, the performance went 202 00:11:58,640 --> 00:12:01,640 up to 98%. How could that be? 203 00:12:01,720 --> 00:12:03,440 I mean, they, they started with 90. 204 00:12:03,640 --> 00:12:06,440 Well, the reason is that they have different knowledge. 205 00:12:06,720 --> 00:12:11,040 The AI had access to much, much more data than any individual 206 00:12:11,040 --> 00:12:14,240 could ever see in their lifetime in terms of different rare skin 207 00:12:14,240 --> 00:12:17,280 conditions. And the doctor had much deeper 208 00:12:17,280 --> 00:12:19,680 knowledge about the the ones that are the most common. 209 00:12:19,680 --> 00:12:22,760 So, you know, and and it's a Doctor Who's calling the shots, 210 00:12:22,760 --> 00:12:28,760 but the doctor takes advantage of this other partner in order 211 00:12:28,760 --> 00:12:31,520 to make better diagnosis. And so this is what what's 212 00:12:31,800 --> 00:12:35,480 laying out now in every area where it's being used by 213 00:12:35,480 --> 00:12:41,880 writers, by people who have to, you know, work on ad copy, 214 00:12:42,640 --> 00:12:45,040 people who have, you know, engineers are trying to solve 215 00:12:45,040 --> 00:12:47,920 problem. By the way, the by far the most 216 00:12:47,920 --> 00:12:51,800 impact that a is had over the last 10 years has been in 217 00:12:51,800 --> 00:12:55,560 science has transformed science, because science is filled with 218 00:12:55,560 --> 00:13:00,720 big data on specific projects that are really very complex, 219 00:13:00,720 --> 00:13:04,440 you know, big particle accelerators, huge, huge amounts 220 00:13:04,440 --> 00:13:07,880 of data that's being generated. And, and it's it's really 221 00:13:07,880 --> 00:13:10,000 transformed the way scientists do science. 222 00:13:10,000 --> 00:13:12,720 And so this is really a harbinger of what to expect in 223 00:13:12,720 --> 00:13:16,320 other areas. Please subscribe to our 224 00:13:16,320 --> 00:13:19,920 newsletter, go to cxotalk.com, subscribe to our newsletter. 225 00:13:20,160 --> 00:13:22,000 And we do have some questions coming in. 226 00:13:22,000 --> 00:13:28,960 So let's jump to LinkedIn and we have a question from Greg 227 00:13:28,960 --> 00:13:37,960 Walters who said he read some somewhere that AGILLMSAI, 228 00:13:37,960 --> 00:13:43,560 digital twins, etcetera visualize actions and processes 229 00:13:43,560 --> 00:13:48,520 like athletes and and more a million hours of study in one 230 00:13:48,520 --> 00:13:50,520 second. Is this correct? 231 00:13:50,640 --> 00:13:56,480 And how does this visualization impact model collapse and does 232 00:13:56,480 --> 00:13:58,720 it? This gets to the heart of of how 233 00:13:58,720 --> 00:14:01,240 you actually create one of these large language models. 234 00:14:01,240 --> 00:14:04,920 You start with a huge amount of data, trillions of words, and 235 00:14:04,920 --> 00:14:10,000 you go through this process of, of, of predicting the next word 236 00:14:10,000 --> 00:14:13,560 and gradually getting to the point where you can with very 237 00:14:13,560 --> 00:14:15,240 high accuracy. And the only way you could have 238 00:14:15,240 --> 00:14:19,480 gotten that where that point is by being able to disambiguate. 239 00:14:19,480 --> 00:14:22,000 Words have multiple meanings and you have to look at the context. 240 00:14:22,440 --> 00:14:25,920 And so it has, it has to create an internal representation of 241 00:14:25,920 --> 00:14:28,840 the semantics, the meaning of the sentence, in order to be 242 00:14:28,840 --> 00:14:30,440 able to understand just the way you do it, right? 243 00:14:30,440 --> 00:14:33,480 You can predict the next word, but you know, you have to 244 00:14:33,480 --> 00:14:35,680 understand what the sentence means before you can do that. 245 00:14:36,680 --> 00:14:40,560 So but that takes a huge amount of time and and very expensive. 246 00:14:40,560 --> 00:14:44,560 The the the latest GPT 4, for example, took two months with 247 00:14:44,560 --> 00:14:49,440 20,000 GP US of graphics processing units and cost $100 248 00:14:49,440 --> 00:14:52,440 million, right. So, but the point is that not 249 00:14:52,440 --> 00:14:57,400 only now can the, can the, the large language model chap GPT 250 00:14:58,920 --> 00:15:03,120 answer new questions, but it can do that like in you press the 251 00:15:03,120 --> 00:15:05,120 button and you get the answer in, you know, one or two 252 00:15:05,120 --> 00:15:06,920 seconds, right? And, and that's shocking. 253 00:15:07,400 --> 00:15:09,880 I, when I first did it, I just couldn't believe this is how 254 00:15:09,880 --> 00:15:12,360 could this be? And the reason is it's 255 00:15:12,360 --> 00:15:14,560 internalized all this information so that it 256 00:15:14,560 --> 00:15:19,040 immediately is able to respond literally, you know, without 257 00:15:19,040 --> 00:15:22,960 actually having to think. In fact, these large, they don't 258 00:15:22,960 --> 00:15:24,920 think, they don't think the way humans think. 259 00:15:25,240 --> 00:15:27,360 And that's one of the things we're going to try to improve. 260 00:15:27,880 --> 00:15:32,480 The next generation is going to actually have internal self 261 00:15:32,480 --> 00:15:35,080 generated activity. Right now, if you stop your 262 00:15:35,080 --> 00:15:38,640 dialogue with ChatGPT, it goes blank. 263 00:15:38,640 --> 00:15:41,680 It doesn't keep thinking about things the way that you do. 264 00:15:41,840 --> 00:15:44,760 If you're on your own without any sensory input, You think 265 00:15:44,760 --> 00:15:47,800 about planning, you think about what happened, you know, in the 266 00:15:47,800 --> 00:15:50,400 past. Chat GDP doesn't do that. 267 00:15:50,400 --> 00:15:51,640 It doesn't have an internal life. 268 00:15:52,080 --> 00:15:57,040 This is from Arsalan Khan and on Twitter. 269 00:15:57,120 --> 00:16:01,840 And he says artificial intelligence is a mirror of 270 00:16:01,920 --> 00:16:08,680 human biases. How can AI and our brain become 271 00:16:08,680 --> 00:16:12,840 less biased quickly? And I I think this leads us 272 00:16:12,840 --> 00:16:16,800 directly into a discussion of the impact of AI on our 273 00:16:16,800 --> 00:16:21,040 workforce and work displacements and all those implications. 274 00:16:21,480 --> 00:16:25,840 The bias could come in because the data are, are incomplete or 275 00:16:26,160 --> 00:16:28,240 skewed. And, and, and, and this came 276 00:16:28,240 --> 00:16:31,560 out, for example, when face recognition was being used. 277 00:16:31,560 --> 00:16:35,160 It was much, much, much more accurate with white faces and 278 00:16:35,160 --> 00:16:36,960 black faces. And it turns out that there were 279 00:16:36,960 --> 00:16:39,560 many fewer black faces that were used in a training set. 280 00:16:39,800 --> 00:16:42,840 So you go back to the training set, you have to curate it. 281 00:16:43,360 --> 00:16:46,040 You have to put, make sure there's, and now we're 282 00:16:46,040 --> 00:16:47,800 discovering it. The higher the quality of the 283 00:16:47,800 --> 00:16:49,920 data, the better the performance in general. 284 00:16:51,320 --> 00:16:52,600 But there's other things you can do. 285 00:16:52,600 --> 00:16:55,960 And this is, this is a little bit more tricky right now. 286 00:16:56,120 --> 00:16:59,760 You know, you give, you have to give a goal because it is, you 287 00:16:59,760 --> 00:17:03,720 know, deep learning networks don't have intrinsic goals the 288 00:17:03,720 --> 00:17:08,000 way that humans have goals. So you have to give it the goal. 289 00:17:08,000 --> 00:17:11,680 And the goal typically is I want to optimize performance on a 290 00:17:11,680 --> 00:17:14,319 particular class of problems. And so you give it a lot of 291 00:17:14,319 --> 00:17:18,200 examples and it, it chugs away and, and it gets better and 292 00:17:18,200 --> 00:17:20,680 better and better and better. And the larger and larger the 293 00:17:20,680 --> 00:17:22,640 network, the, the better and better it gets. 294 00:17:23,520 --> 00:17:26,720 But the point though, is that it, it may not be aligned with a 295 00:17:26,720 --> 00:17:30,040 lot of other values that you have, like fairness, right? 296 00:17:30,880 --> 00:17:33,960 And So what do you do? Well, you're going to have to 297 00:17:34,400 --> 00:17:38,080 add fairness as one of the goals that it has. 298 00:17:38,080 --> 00:17:42,880 And now you have a problem. OK, I want maximum performance, 299 00:17:42,880 --> 00:17:44,920 but I also want maximum fairness. 300 00:17:44,920 --> 00:17:48,400 I can't have both, right? So I have to wait them. 301 00:17:48,400 --> 00:17:52,960 I have to decide, well, half and half or maybe 90% performance 302 00:17:52,960 --> 00:17:56,600 and 10% fairness, you know, that that's a decision has to be made 303 00:17:56,600 --> 00:17:58,720 explicitly. And you know, we're somehow 304 00:17:58,720 --> 00:18:01,880 we're making it implicitly. And when we deal with the world, 305 00:18:02,400 --> 00:18:06,880 you know, given the cultural biases that we have now, here's 306 00:18:06,880 --> 00:18:09,680 something about bias, though, and, and I just want to put this 307 00:18:09,680 --> 00:18:12,000 to you, OK, I'm going to ask a question of you and your 308 00:18:12,000 --> 00:18:16,880 audience, OK? You know, humans are definitely 309 00:18:16,880 --> 00:18:21,520 biased, right? We see this all time and LO, Ms. 310 00:18:21,520 --> 00:18:23,560 reflect that to some extent because they're trained on human 311 00:18:23,560 --> 00:18:26,400 data. Here's the question, which do 312 00:18:26,400 --> 00:18:30,640 you think are going to be easier to fix, the LLM or the human? 313 00:18:30,960 --> 00:18:33,880 I would. Assume the human is easier to 314 00:18:33,880 --> 00:18:40,280 fix because with the human you have control in quotes over one, 315 00:18:40,520 --> 00:18:45,600 whereas to fix bias in the LLM you need to adjust a very large 316 00:18:45,600 --> 00:18:47,760 data set. Actually, I'd be curious to know 317 00:18:47,760 --> 00:18:51,120 what your audience thinks. Folks, you heard the question. 318 00:18:51,520 --> 00:18:55,480 In my experience, I've known a lot of different people and very 319 00:18:55,480 --> 00:18:57,360 rare that they change their minds about it. 320 00:18:57,680 --> 00:19:00,600 They're really fixed in their ways. 321 00:19:00,600 --> 00:19:04,240 You know, as you get older, I mean, maybe younger children, 322 00:19:04,240 --> 00:19:06,880 OK, there, there, there's some hope there of, of changing their 323 00:19:06,880 --> 00:19:08,120 by. But you come into the world, you 324 00:19:08,120 --> 00:19:10,920 don't know what the language is. You don't know what the values 325 00:19:10,920 --> 00:19:13,120 are of the culture. And you learn through 326 00:19:13,120 --> 00:19:15,760 reinforcement learning, by the way, you know, what's, what's 327 00:19:16,320 --> 00:19:19,720 things that you, that you should, how, how to deal with 328 00:19:19,720 --> 00:19:23,400 relationships, how, how to value, you know, you know, 329 00:19:23,400 --> 00:19:27,440 people's advice, think things that you know, that are 330 00:19:27,520 --> 00:19:32,760 different in different cultures, how you deal with, you know, 331 00:19:32,760 --> 00:19:35,200 truth and so forth. It's it's different, it's not 332 00:19:35,200 --> 00:19:41,360 the same and and you learn. And Greg Walters on LinkedIn 333 00:19:41,600 --> 00:19:44,240 agrees with you. He comes back and he said it's 334 00:19:44,240 --> 00:19:48,040 the LLM that will be easier to. That's because we engineered it. 335 00:19:48,040 --> 00:19:49,880 We can fix it. But how? 336 00:19:50,040 --> 00:19:56,240 How do you fix that bias given there are so many levels of 337 00:19:56,760 --> 00:20:00,520 insidious? OK, OK, OK, so he he in my book, 338 00:20:00,560 --> 00:20:04,880 you know, chat GDP and the future of I, I actually laid out 339 00:20:04,880 --> 00:20:07,840 because I there's a there's a lot in the book that is is is 340 00:20:07,840 --> 00:20:11,240 really inspired by the brain. And by the way, there's a 341 00:20:11,520 --> 00:20:14,480 revolution going on in neuroscience that is equally 342 00:20:14,480 --> 00:20:16,800 important as what's happening in AI. 343 00:20:16,800 --> 00:20:19,240 And this has to do with the brain initiative, Obama's brain 344 00:20:19,240 --> 00:20:22,120 initiative. It's been 10 years on and we 345 00:20:22,120 --> 00:20:27,640 have made enormous strides in understanding the mechanisms 346 00:20:27,640 --> 00:20:31,640 underlying your brain. And so we're, we're going to 347 00:20:31,640 --> 00:20:35,680 actually be able to really, you know, get, get, come up again 348 00:20:35,680 --> 00:20:39,200 with a much better explanation for the decisions that we make. 349 00:20:39,920 --> 00:20:43,920 But, but, but here, here's, you know, when, when we're dealing 350 00:20:43,920 --> 00:20:50,200 with, as I told you with for humans, you use reinforcement 351 00:20:50,200 --> 00:20:51,840 learning. So here's what has to happen in 352 00:20:51,840 --> 00:20:55,200 AI instead of just training the network at the beginning. 353 00:20:55,200 --> 00:20:57,200 And then that's it. You, what you're going to have 354 00:20:57,200 --> 00:20:59,880 to do is use reinforcement learning all along. 355 00:21:00,120 --> 00:21:02,960 We need lifelong learning where we have reinforcement learning 356 00:21:02,960 --> 00:21:06,800 going hand in hand with the, the, the declarative, it's 357 00:21:06,800 --> 00:21:08,480 called the declarative knowledge system. 358 00:21:09,000 --> 00:21:12,960 And, and so we have to basically treat the LLM like a child, 359 00:21:13,560 --> 00:21:16,640 right, You know, and, and punish the child when it does say 360 00:21:16,640 --> 00:21:19,520 something wrong, does something wrong, and that how else is it 361 00:21:19,520 --> 00:21:21,720 going to learn that right? We have to align it with our 362 00:21:21,720 --> 00:21:24,000 values. And, and, and that's beginning 363 00:21:24,000 --> 00:21:25,800 to happen. And I know a couple of companies 364 00:21:25,800 --> 00:21:29,000 already used it, not for this particular purpose, but for 365 00:21:29,000 --> 00:21:30,640 example, something called chain of thought. 366 00:21:30,920 --> 00:21:33,160 When you want to solve a problem, you break into pieces 367 00:21:33,400 --> 00:21:34,680 and you solve each one separately. 368 00:21:34,680 --> 00:21:37,160 Mathematicians do this all time when they're proving a theorem. 369 00:21:37,640 --> 00:21:41,200 But the LLMS, if you ask them, they just jump to the answer and 370 00:21:41,200 --> 00:21:42,760 sometimes they're right, sometimes they're wrong. 371 00:21:43,200 --> 00:21:45,960 But now you can actually train using reinforcement learning, 372 00:21:46,400 --> 00:21:50,080 this habit of breaking into pieces and solving each one 373 00:21:50,080 --> 00:21:51,640 separately. And now it's gone. 374 00:21:51,720 --> 00:21:54,560 The performance has gone on these mathematical problems from 375 00:21:54,560 --> 00:21:57,480 20% to 80%. So this is an example, you know, 376 00:21:57,600 --> 00:21:59,000 you're using reinforcement learning. 377 00:21:59,000 --> 00:22:04,280 It's actually in part a, a way of a way of solving problems and 378 00:22:04,280 --> 00:22:07,200 actually a method that could help not not just solve 379 00:22:07,200 --> 00:22:10,440 mathematical problems, but any problem that requires, you know, 380 00:22:10,480 --> 00:22:15,560 in, in in manufacturing, in figuring out how to reconfigure 381 00:22:15,560 --> 00:22:18,840 your office, all of that. Should we should be able to 382 00:22:19,080 --> 00:22:21,040 help. We should be able to explain, 383 00:22:21,920 --> 00:22:24,080 you know, through a sequence of actions, how you're going to get 384 00:22:24,080 --> 00:22:27,160 to some goal. And in answer to your question 385 00:22:27,440 --> 00:22:32,560 to the audience of which will be easier to remove or address bias 386 00:22:32,800 --> 00:22:38,400 in the LLM or in people, Isaac Sacolic comes back and he says 387 00:22:38,400 --> 00:22:40,480 it's easier to fix AI than humans. 388 00:22:40,480 --> 00:22:44,080 People don't change belief systems easily, and data and 389 00:22:44,080 --> 00:22:47,080 weights can change in an AI. The real issue is understanding 390 00:22:47,560 --> 00:22:51,040 what values the data scientists embedded in their training. 391 00:22:51,400 --> 00:22:57,000 So he's in agreement with you. And Arsalan Khan is also in 392 00:22:57,000 --> 00:22:59,200 agreement. Everybody's against what I said. 393 00:22:59,760 --> 00:23:06,000 Arsalan Khan says LLMS might be easier to change since the input 394 00:23:06,160 --> 00:23:10,120 is from multiple people, whereas people rarely changed. 395 00:23:10,280 --> 00:23:15,680 Exactly your point. But now let's talk about the 396 00:23:15,680 --> 00:23:23,320 implications of all of this on work, the workforce, jobs. 397 00:23:23,560 --> 00:23:26,560 Thoughts on that? We are at the stage that the 398 00:23:26,560 --> 00:23:30,240 Wright brothers were at the beginning of aviation 100 years 399 00:23:30,240 --> 00:23:32,160 ago. The very first flight by the 400 00:23:32,520 --> 00:23:35,800 Wright brothers was 10 feet up and 100 feet forward. 401 00:23:36,360 --> 00:23:39,480 That was it. They were inspired by how birds 402 00:23:39,480 --> 00:23:42,560 glide by the airfoil. They were inspired by the 403 00:23:42,560 --> 00:23:46,600 lightness of the wings, and that was why they use canvas. 404 00:23:47,160 --> 00:23:51,400 But the most difficult part of making aviation safe was 405 00:23:51,400 --> 00:23:56,040 figuring out how to regulate, you know, and how to control the 406 00:23:56,040 --> 00:23:58,760 direction you're going and, and regulate it so that you don't 407 00:23:58,760 --> 00:24:01,320 crash. That was the difficult problem. 408 00:24:02,320 --> 00:24:04,720 Well, we're, we're living through that right now, except 409 00:24:04,720 --> 00:24:06,200 that maybe it won't take 100 years. 410 00:24:06,200 --> 00:24:08,960 It'll only take, you know, 1020 years. 411 00:24:09,640 --> 00:24:12,440 So we're, we're, we're, we're just at the very beginning of 412 00:24:12,440 --> 00:24:13,960 that process. And what's going to happen over 413 00:24:13,960 --> 00:24:18,480 the next 10 years near term is incremental advances and that's 414 00:24:18,480 --> 00:24:20,120 already happening. You see all these language 415 00:24:20,120 --> 00:24:23,920 models, there's now a dozens of them out there with different 416 00:24:24,000 --> 00:24:28,000 capabilities and, you know, focused on different problems 417 00:24:28,000 --> 00:24:30,960 that have to be solved. So, you know, we're, we're just 418 00:24:30,960 --> 00:24:34,280 exploring that and, and one of the directions that we're going 419 00:24:34,280 --> 00:24:37,080 is actually not bigger and bigger because we've run out of 420 00:24:37,080 --> 00:24:43,440 data, but smaller and smaller with quality data, with data 421 00:24:43,440 --> 00:24:47,160 that's more specific to your particular company, to your 422 00:24:47,160 --> 00:24:50,720 particular profession. And that now is, is going to 423 00:24:51,000 --> 00:24:54,040 shift the balance away from these big high tech companies 424 00:24:54,200 --> 00:24:58,480 toward, you know, the enterprise, the companies, the 425 00:24:58,480 --> 00:25:01,720 ones that are actually doing, you know, the, the, by far the 426 00:25:01,720 --> 00:25:06,800 most important job in society, which was, you know, making 427 00:25:06,800 --> 00:25:10,960 things and getting things to work and, and dealing with the 428 00:25:10,960 --> 00:25:13,400 complexity of the world. And that, that could be done. 429 00:25:14,360 --> 00:25:18,480 This, this is something that is, is, is, is not going to take 430 00:25:18,480 --> 00:25:21,560 just five years. It's not going to take 10 years 431 00:25:21,960 --> 00:25:25,200 because it takes is you have to complete reorganization of the 432 00:25:25,200 --> 00:25:30,480 way that the, the, your office and your company is organized. 433 00:25:30,920 --> 00:25:34,320 So right now, for example, I'll just give one example. 434 00:25:35,840 --> 00:25:38,080 You know, you have databases all over the place, especially if 435 00:25:38,080 --> 00:25:40,920 you're an international company and you know, how do you 436 00:25:40,920 --> 00:25:43,440 integrate across databases? Well, I mean, you have an IT 437 00:25:43,440 --> 00:25:46,200 person who has ways of doing that and, but you know, to 438 00:25:46,200 --> 00:25:49,240 answer your question, that requires a lot of comparison 439 00:25:49,240 --> 00:25:54,560 and, and, you know, accretion of, of, of, of different 440 00:25:54,560 --> 00:25:57,760 sources. That's, that's going to take a 441 00:25:57,760 --> 00:25:59,640 long time. But it one of the things that 442 00:25:59,640 --> 00:26:01,960 LMS are really good at it because it's already done that 443 00:26:01,960 --> 00:26:04,800 for the database of the world. Why not for your company? 444 00:26:04,920 --> 00:26:06,920 All of those databases could be integrated. 445 00:26:07,280 --> 00:26:09,760 And now when you want to answer a question, you don't go to the 446 00:26:09,760 --> 00:26:13,200 IT guy. You ask your, your ChatGPT. 447 00:26:13,200 --> 00:26:14,840 Well, you'd call it something else, right? 448 00:26:14,840 --> 00:26:19,960 You know, it would be called chat IBM, you know, and, and 449 00:26:19,960 --> 00:26:22,440 that way you're, you're going to be able to have access to 450 00:26:22,440 --> 00:26:25,760 instantly to all kinds of important questions that you can 451 00:26:25,760 --> 00:26:28,760 answer that may help you make decisions. 452 00:26:28,760 --> 00:26:31,080 This is if you're, you know, of course, a head of a company. 453 00:26:31,240 --> 00:26:32,880 What about if you're in the middle of the company? 454 00:26:33,320 --> 00:26:37,240 Now, one of the things to keep in mind is that these tools 455 00:26:37,640 --> 00:26:41,040 require training. You know, chat GDP 456 00:26:41,080 --> 00:26:43,560 out-of-the-box. You can ask the questions, but 457 00:26:43,560 --> 00:26:46,200 the answers you get back are often not the most useful. 458 00:26:46,200 --> 00:26:49,320 That's because like any tool, you have to learn how to use the 459 00:26:49,320 --> 00:26:52,120 tool. And you know, there's a, a whole 460 00:26:52,120 --> 00:26:56,680 group of people now that call themselves prompt engineers. 461 00:26:56,680 --> 00:26:57,920 And what, what, what does that mean? 462 00:26:57,920 --> 00:27:03,120 It means that they know how, how to use the prompt in order to be 463 00:27:03,120 --> 00:27:06,800 able to more quickly get useful answers out. 464 00:27:06,800 --> 00:27:11,320 And, and in my book, I go into this in quite some detail that 465 00:27:11,320 --> 00:27:15,800 may help, that may help them. And, and the other thing is, is 466 00:27:15,800 --> 00:27:18,840 it's, it's really interesting. It's the, you know, if you 467 00:27:18,840 --> 00:27:23,760 really want to incentivize people, you, you know, you what 468 00:27:23,760 --> 00:27:27,000 you have to do, dealing with chatty P is fine. 469 00:27:27,000 --> 00:27:29,240 People do it on their own. People just doing it because 470 00:27:29,240 --> 00:27:32,360 it's a lot of fun, right? Well, once you, once they get 471 00:27:32,360 --> 00:27:35,760 trained on doing this, this is going to help with productivity. 472 00:27:35,760 --> 00:27:38,960 It's, it's, it's already doing that in the many areas, like I 473 00:27:38,960 --> 00:27:42,160 gave the example of medicine, but it, it's, it's going to help 474 00:27:42,160 --> 00:27:45,360 people in ways that nobody ever even imagined, right? 475 00:27:45,360 --> 00:27:50,440 I mean, this is, it's, it's, this is something that like, you 476 00:27:50,440 --> 00:27:53,200 know, what was the killer app for personal computers, right? 477 00:27:53,200 --> 00:27:58,400 It, it wasn't, you know, simply typing and, and you know, the 478 00:27:58,960 --> 00:28:02,160 letters, it was Lotus 123. Do you remember that? 479 00:28:02,160 --> 00:28:06,160 It was a spreadsheet and, and that suddenly that was something 480 00:28:06,160 --> 00:28:10,360 that's very useful that could be used by not just individuals, 481 00:28:10,360 --> 00:28:14,320 but by companies and organizing all the data and so forth. 482 00:28:14,320 --> 00:28:18,040 And, and that has become central to the, to the way that now we 483 00:28:18,040 --> 00:28:23,120 all deal with large data sets as we, we have them in these super, 484 00:28:24,360 --> 00:28:27,000 you know, you know, the spreadsheets now are, are, are 485 00:28:27,000 --> 00:28:29,320 much better design than Louis 123. 486 00:28:29,560 --> 00:28:32,680 But the same thing is going on right now is that we're, we're, 487 00:28:32,880 --> 00:28:37,000 we're beginning to develop the tools that are specific and in 488 00:28:37,000 --> 00:28:39,280 surprising ways. Let me give you one example of 489 00:28:39,280 --> 00:28:42,360 something that surprised me. It's in my, and again, it's in 490 00:28:42,360 --> 00:28:46,160 my book. So a lot of people have mental 491 00:28:46,160 --> 00:28:50,440 problems, you know, they have anxiety, they have phobias and 492 00:28:50,440 --> 00:28:52,040 so forth. And, and you know, and, and 493 00:28:52,040 --> 00:28:55,360 worse and mental disorders, but you know that what do they do? 494 00:28:55,360 --> 00:29:00,360 They go to a psychiatrist or a cognitive therapist and you 495 00:29:00,360 --> 00:29:04,160 know, they, they have to make their appointment weeks ahead of 496 00:29:04,160 --> 00:29:07,880 time. And then, you know, they go and 497 00:29:07,880 --> 00:29:10,440 then, you know, they have an hour and then they get a big 498 00:29:10,440 --> 00:29:14,400 bill right now. You know, this is, this is, you 499 00:29:14,400 --> 00:29:17,240 know, it, it helps, it really, it turns out cognitive therapy 500 00:29:17,240 --> 00:29:20,000 is as good as taking a pill. It really is. 501 00:29:21,200 --> 00:29:23,280 It's and, and it's complementary, by the way. 502 00:29:24,520 --> 00:29:28,480 Now they did a test. This is a scientific study. 503 00:29:29,120 --> 00:29:34,480 If people are given the choice between an AI psychiatrist or a 504 00:29:34,480 --> 00:29:36,240 human psychiatrist, what would they choose? 505 00:29:36,240 --> 00:29:38,400 What do you think, Michael? What do you think? 506 00:29:38,760 --> 00:29:44,000 I will say that they choose the AI over the psychiatrist you'd. 507 00:29:44,000 --> 00:29:46,040 Figure this out from my first trick question. 508 00:29:48,000 --> 00:29:51,320 Yeah you're absolutely right. But majority of, of humans feel 509 00:29:51,320 --> 00:29:56,720 more comfortable talking about, you know, personal things that 510 00:29:56,720 --> 00:29:58,160 it might be a little embarrassing, right? 511 00:29:58,160 --> 00:30:00,440 If they're talking to another human who are, you know, humans 512 00:30:00,440 --> 00:30:03,640 are very judgmental, right? I mean, but the AI, you know, 513 00:30:03,640 --> 00:30:06,040 it, it's not going to judge you away. 514 00:30:06,040 --> 00:30:09,600 It doesn't have any internal care, but you know, it's going 515 00:30:09,600 --> 00:30:13,560 to give you respond in a way that is objective and so forth. 516 00:30:14,960 --> 00:30:19,080 So you know and you know this is this already happened back at 517 00:30:19,080 --> 00:30:23,040 MIT in the early days of AI when someone put a very simple 518 00:30:23,040 --> 00:30:25,560 program that it did very simple thing that just repeated the 519 00:30:25,560 --> 00:30:27,280 question. Eliza. 520 00:30:27,720 --> 00:30:32,960 Eliza And, you know, a Weisbaum Weisenbaum discovered that his 521 00:30:32,960 --> 00:30:35,000 secretary is using it during the lunch break. 522 00:30:35,000 --> 00:30:39,200 You know, it's like, and you know, it didn't, she didn't care 523 00:30:39,200 --> 00:30:41,840 that it was a machine as in a very simple minded machine. 524 00:30:41,840 --> 00:30:44,040 You know, she did it because it kind of helped. 525 00:30:44,040 --> 00:30:46,640 It got her to think about things, you know, explicitly. 526 00:30:46,840 --> 00:30:48,480 So there you go. I mean, that's that's 527 00:30:48,480 --> 00:30:50,720 unexpected, right? There's a lot of unintended 528 00:30:51,040 --> 00:30:52,800 consequences. And by the way, not all good. 529 00:30:52,800 --> 00:30:54,480 There's going to be bad ones too. 530 00:30:54,480 --> 00:30:57,120 And so that's why we have to really be careful about, you 531 00:30:57,120 --> 00:31:02,080 know, what, what to expect. You have just been describing 532 00:31:02,280 --> 00:31:08,560 what I would call incremental changes of efficiency, right? 533 00:31:08,560 --> 00:31:14,040 We now have a critical thinking or an information partner to 534 00:31:14,040 --> 00:31:17,280 help us improve our writing to research and so forth. 535 00:31:17,600 --> 00:31:25,120 But what about the more that the deeper implications and more 536 00:31:25,760 --> 00:31:30,960 foundational changes that this may drive in society and in in 537 00:31:30,960 --> 00:31:32,880 the workplace? Any any thoughts on that? 538 00:31:33,320 --> 00:31:35,480 In the short room, you're not going to lose your job. 539 00:31:35,880 --> 00:31:39,720 Your job is going to change and it's going to change because you 540 00:31:39,720 --> 00:31:41,760 have these new tools. You're going to have to learn 541 00:31:41,760 --> 00:31:44,400 new skills. And that's why it's really 542 00:31:44,400 --> 00:31:48,080 important that we have within companies and, and it turns out 543 00:31:48,080 --> 00:31:52,200 that we have a, a ways of doing that that could help companies. 544 00:31:52,920 --> 00:31:56,240 This is called massive Open online courses and they're free 545 00:31:56,240 --> 00:31:58,120 and there's a lot of out there now there's like, you know, 546 00:31:58,120 --> 00:32:02,000 10,000 and I have one myself called learning how to learn. 547 00:32:02,880 --> 00:32:05,640 And this is with Barbara Oakley. And what we realized was a lot 548 00:32:05,640 --> 00:32:10,400 of people, especially 25 to 35, half of them are college 549 00:32:10,400 --> 00:32:13,360 educated, are in the workforce and now they need new skills. 550 00:32:13,360 --> 00:32:16,720 And it's harder to learn when you're in that position because 551 00:32:16,720 --> 00:32:20,440 you can't go back to school and you have mortgage and children. 552 00:32:20,440 --> 00:32:23,560 And so you know, you want to be able to be more efficient. 553 00:32:23,560 --> 00:32:26,240 We teach you how to be more efficient through what we know 554 00:32:26,240 --> 00:32:27,880 about the brain, how the brain learns. 555 00:32:28,360 --> 00:32:31,800 In any case, they're there, but there are all kinds of mooks out 556 00:32:31,800 --> 00:32:36,520 there about how to use the new tools of and the idea I think is 557 00:32:36,560 --> 00:32:42,880 for people now who have a job are going to be using your tools 558 00:32:42,880 --> 00:32:45,080 more efficiently. But gradually the job is going 559 00:32:45,080 --> 00:32:47,560 to morph. It's it's going to be the 560 00:32:47,560 --> 00:32:50,240 boundaries between jobs may change in terms of, you know, 561 00:32:50,240 --> 00:32:54,000 what people can do and integrating more often rather 562 00:32:54,000 --> 00:32:56,680 than having stacked layers and so forth. 563 00:32:57,360 --> 00:32:59,600 You know, it's, it's impossible to predict the future. 564 00:32:59,600 --> 00:33:03,640 Let me ask you, OK, Michael, you were around in the 90s, right? 565 00:33:04,080 --> 00:33:05,440 Yes. OK. 566 00:33:05,600 --> 00:33:08,120 You remember when the Internet went public, right? 567 00:33:08,720 --> 00:33:11,280 Yeah. Could you have predicted what 568 00:33:11,280 --> 00:33:13,800 influence the Internet would have on every aspect of your 569 00:33:13,800 --> 00:33:15,840 life today? Absolutely not. 570 00:33:15,840 --> 00:33:17,280 Not at that time. OK. 571 00:33:17,280 --> 00:33:19,600 Well, we have another technology that's that's going to have a 572 00:33:19,600 --> 00:33:23,400 equally important impact up the road that we can't predict. 573 00:33:23,640 --> 00:33:28,680 It's, it's just, it has, it has so many different effects, so 574 00:33:28,680 --> 00:33:33,520 many different parts of society. But Terry, I did not invent or 575 00:33:33,520 --> 00:33:38,000 discover important aspects, parts of the that underlie the 576 00:33:38,400 --> 00:33:42,640 Internet, but you did when it comes to deep learning. 577 00:33:42,640 --> 00:33:47,880 And so therefore I look to you as the the the prophet and the 578 00:33:47,880 --> 00:33:52,200 seer of what may come. The reality is that I'm actually 579 00:33:53,480 --> 00:33:56,720 probably no more capable than you are. 580 00:33:56,720 --> 00:33:59,880 In fact, maybe less capable of of predicting the future. 581 00:34:00,800 --> 00:34:05,800 You know, the Internet pioneers back then. 582 00:34:05,800 --> 00:34:10,679 Let me give you an example, OK. You know, Google, when it 583 00:34:10,679 --> 00:34:13,600 started had a motto, do no evil. Do you remember that? 584 00:34:14,360 --> 00:34:16,280 Yeah, absolutely. And, and, you know, that they 585 00:34:16,280 --> 00:34:19,280 were very idealistic. They said this is going to 586 00:34:19,280 --> 00:34:22,159 democratize information. Everybody is going to have 587 00:34:22,159 --> 00:34:25,159 access to information and their voices will be heard. 588 00:34:25,639 --> 00:34:31,000 OK, well, could they have predicted how that will play 589 00:34:31,000 --> 00:34:32,840 out? And here we are now we're we're 590 00:34:32,840 --> 00:34:37,400 dealing with, you know, misinformation, fake news, echo 591 00:34:37,400 --> 00:34:39,000 chambers. I mean, this is like, you know, 592 00:34:39,000 --> 00:34:40,679 who could have predicted that, right? 593 00:34:40,679 --> 00:34:43,600 I mean, this is something that even the experts didn't predict. 594 00:34:43,920 --> 00:34:47,880 The enthusiasm of young people before they became billionaires. 595 00:34:47,880 --> 00:34:49,159 That's right. That's right. 596 00:34:49,159 --> 00:34:52,880 So I'm a young, I was once a good person in any case. 597 00:34:54,960 --> 00:34:58,080 Yeah, we, we, we, we are, you know, I'm, I'm actually an 598 00:34:58,080 --> 00:35:00,280 optimist. I, I actually think that we will 599 00:35:00,280 --> 00:35:03,840 get it sorted out, but I know it's not going to be easy and 600 00:35:03,840 --> 00:35:06,200 it's going to take a long time. And and that's true of all 601 00:35:06,200 --> 00:35:07,960 technologies. It's not like it, this is 602 00:35:07,960 --> 00:35:10,120 different. It's just, it's just that it's 603 00:35:10,760 --> 00:35:17,360 so new that we really are just beginning to, you know, you 604 00:35:17,440 --> 00:35:21,920 know, trying to beginning to understand the capabilities it's 605 00:35:21,920 --> 00:35:24,840 moving target, it's incrementally improving there. 606 00:35:25,160 --> 00:35:28,080 There may be a breakthrough and it may not be, it may be, you 607 00:35:28,080 --> 00:35:30,720 know, I don't know, 10 years, maybe 20 years. 608 00:35:32,040 --> 00:35:34,320 And I think it's going to be the breakthrough, I think is going 609 00:35:34,320 --> 00:35:38,080 to be when you put something which call is called System 2 610 00:35:38,080 --> 00:35:39,560 into these large language models. 611 00:35:39,560 --> 00:35:43,120 System 1 is what we have right now, which is basically just 612 00:35:43,120 --> 00:35:46,640 trained to, to produce, you know, generate the generative AI 613 00:35:46,640 --> 00:35:50,360 one word after the next. But this, this system too, is 614 00:35:50,360 --> 00:35:52,960 the one I alluded to earlier, which is cell generated 615 00:35:52,960 --> 00:35:55,600 activity. And, and we, we actually know 616 00:35:55,600 --> 00:35:57,080 how that's done in the human brain. 617 00:35:57,080 --> 00:35:59,440 And so it's just going to be a matter of time before that's 618 00:36:00,480 --> 00:36:04,320 transplanted into AI. You know it, but it's not going 619 00:36:04,320 --> 00:36:07,320 to take place in, in, in 5 or 10 years. 620 00:36:07,320 --> 00:36:09,920 We're talking here for probably, you know, decades out. 621 00:36:10,320 --> 00:36:17,880 Chris Peterson on Twitter says the current hype based AI takes 622 00:36:17,920 --> 00:36:21,640 vastly more resource inputs per question than a human brain 623 00:36:21,640 --> 00:36:23,880 takes for 24 hours of doing everything. 624 00:36:24,840 --> 00:36:28,200 Can you talk about the sustainability, the power 625 00:36:28,200 --> 00:36:30,880 requirements, the resource requirements? 626 00:36:31,080 --> 00:36:35,480 You alluded to this earlier in your discussion of the trade off 627 00:36:35,480 --> 00:36:41,760 between completeness versus rooting out bias and the input 628 00:36:41,760 --> 00:36:43,480 cost and time associated with that. 629 00:36:43,880 --> 00:36:48,480 We have to set priorities. It can't things have been the 630 00:36:48,480 --> 00:36:51,000 cost of things have is gone through the roof and with that 631 00:36:51,000 --> 00:36:54,760 cannot continue. And I also alluded to the fact 632 00:36:54,760 --> 00:36:57,240 that the next generation they're going to be smaller language 633 00:36:57,240 --> 00:37:00,720 models with higher quality data. That's One Direction. 634 00:37:00,880 --> 00:37:04,240 The other direction is hardware and I have old chapter on this 635 00:37:04,240 --> 00:37:06,320 in my book, right? If you're interested go, you 636 00:37:06,320 --> 00:37:11,200 should go there. So the human brain is able to 637 00:37:11,200 --> 00:37:16,720 function at very high levels with 20 watts of power, some a 638 00:37:16,720 --> 00:37:18,400 little brighter than others, right? 639 00:37:18,720 --> 00:37:23,440 But you know that's much less than the the gigawatts that are 640 00:37:23,440 --> 00:37:27,040 out there being used right now to answer your questions. 641 00:37:27,840 --> 00:37:30,320 So there's clearly a huge, huge imbalance. 642 00:37:30,320 --> 00:37:33,920 And that's because the technology that nature uses is, 643 00:37:33,920 --> 00:37:36,320 is based, is gone down to the molecular level. 644 00:37:36,320 --> 00:37:40,240 It's really, really advanced. But you know, we're, we're 645 00:37:40,240 --> 00:37:43,360 actually, there's a hundreds of companies out there that are 646 00:37:43,360 --> 00:37:46,200 building more efficient technology that is going to make 647 00:37:47,280 --> 00:37:53,960 lower power much more the, the architecture that you need for 648 00:37:54,200 --> 00:37:57,640 these deep learning networks is completely different from the 649 00:37:57,640 --> 00:38:00,320 von Neumann architecture that is in your PC. 650 00:38:00,960 --> 00:38:06,000 That architecture does 1 instruction at a time and it's a 651 00:38:06,000 --> 00:38:08,520 has to fetch information from memory, which is separate. 652 00:38:08,760 --> 00:38:14,440 Well, in your brain, the memory and the processors are one in 653 00:38:14,440 --> 00:38:16,840 the same. They're, they're integrated and 654 00:38:16,880 --> 00:38:18,520 there are a lot more processors, right? 655 00:38:18,520 --> 00:38:20,920 This is all the neurons. You have 100 billion neurons in 656 00:38:20,920 --> 00:38:23,680 your brain. And now we're up to, you know, 657 00:38:23,960 --> 00:38:29,760 it, it, you know, literally 100 trillion of synapses in your 658 00:38:29,760 --> 00:38:33,960 brain, we're up to a trillion weights in these networks. 659 00:38:33,960 --> 00:38:37,240 So we're still like only 1% of the brain even just in raw 660 00:38:37,240 --> 00:38:40,080 power. But here's the beauty is that we 661 00:38:40,080 --> 00:38:43,920 can special purpose chips that could be much more efficient 662 00:38:44,480 --> 00:38:47,040 and, and and that will take us down by a couple orders of 663 00:38:47,040 --> 00:38:49,680 magnitude over the next 10 years already going on. 664 00:38:49,680 --> 00:38:51,600 And this has happened to GP us are a good example. 665 00:38:51,600 --> 00:38:55,400 That is A2 orders of magnitude when they put GPT, when they 666 00:38:55,720 --> 00:39:01,680 transplanted these original deep learning networks onto GPUs, it 667 00:39:01,680 --> 00:39:03,400 was tours of magnitude more efficient. 668 00:39:03,400 --> 00:39:06,720 And the reason is that they have thousands of processing units 669 00:39:06,760 --> 00:39:10,400 just like all the neurons working in parallel exchanging 670 00:39:10,400 --> 00:39:12,880 information. So that was that was a big step. 671 00:39:12,880 --> 00:39:15,080 That was a huge jump. And that there's an inflection. 672 00:39:15,080 --> 00:39:18,160 The the Moore's law went from doubling every two years to 673 00:39:18,160 --> 00:39:20,200 doubling every two months, literally. 674 00:39:20,400 --> 00:39:22,600 I mean, this was a huge, huge change. 675 00:39:22,920 --> 00:39:24,800 Now that's that's going to happen again. 676 00:39:24,800 --> 00:39:28,640 And it's, it's, it's, it's going to become probably another two 677 00:39:28,640 --> 00:39:33,440 orders of magnitude or three, possibly with a, a new class of 678 00:39:33,440 --> 00:39:36,440 computing called neuromorphic engineering. 679 00:39:36,440 --> 00:39:39,120 What is that? Well, it turns out that if if 680 00:39:39,120 --> 00:39:42,520 you take your digital chip, which are basically 0 ones, 681 00:39:42,520 --> 00:39:46,480 right, and run them near 0 where things are analog, by the way, 682 00:39:46,480 --> 00:39:48,360 digital chips are actually analog when you look at the 683 00:39:48,360 --> 00:39:51,240 actual transistors, but they run into the rail. 684 00:39:52,240 --> 00:39:55,160 But if you work down there, and this is Carver B, that Caltech 685 00:39:55,160 --> 00:39:58,040 actually understood this, that you can actually take advantage 686 00:39:58,040 --> 00:40:02,480 of that processing at that low level for doing crude multiplies 687 00:40:02,480 --> 00:40:06,000 and adds much more efficiently by by like two or three orders 688 00:40:06,000 --> 00:40:07,960 of magnitude. And now that's mature 689 00:40:07,960 --> 00:40:09,480 technology. And so that's going to come 690 00:40:09,480 --> 00:40:12,640 online over the next 10 years. So see, what's going to happen 691 00:40:12,640 --> 00:40:15,320 is just like, you know, Moore's laws, that the technology gets 692 00:40:15,320 --> 00:40:17,680 more and more efficient. And, and we're, we're just at 693 00:40:17,680 --> 00:40:21,760 the beginning of that because we, you know, we realize now 694 00:40:21,760 --> 00:40:24,560 that this architecture may be more important than, than the 695 00:40:24,920 --> 00:40:27,800 von Neumann architecture for solving many, many kinds of 696 00:40:27,800 --> 00:40:29,680 problems that before, you know, we couldn't. 697 00:40:30,040 --> 00:40:36,240 This is from Isaac Sukolic and Isaac is a big time CIO 698 00:40:36,320 --> 00:40:39,000 influencers. A lot of CIOs listen to him and 699 00:40:39,000 --> 00:40:42,960 he says he liked your healthcare example. 700 00:40:43,400 --> 00:40:49,320 It may be hard to benchmark human versus human plus AI 701 00:40:49,720 --> 00:40:53,040 versus AI and other diagnostic areas. 702 00:40:53,320 --> 00:41:00,280 What are options to validate accuracy and build trust with AI 703 00:41:00,280 --> 00:41:02,120 recommendations? I think that's a really 704 00:41:02,120 --> 00:41:05,000 important question. That's productivity and, and, 705 00:41:05,000 --> 00:41:06,480 and that's the bottom line actually. 706 00:41:06,480 --> 00:41:10,720 You know, for a long time, computers were you the, the, the 707 00:41:10,840 --> 00:41:14,800 whole business world and, and, and, and public are investing in 708 00:41:14,800 --> 00:41:17,200 digital computers and productivity didn't change very 709 00:41:17,200 --> 00:41:18,440 much. It wasn't until you connected 710 00:41:18,440 --> 00:41:21,040 them together that they could exchange information. 711 00:41:21,240 --> 00:41:26,280 That's when things took off. Let me go back to just answer 712 00:41:26,280 --> 00:41:29,240 the question in a concrete way. Let's go back to healthcare. 713 00:41:30,080 --> 00:41:33,680 So, you know, you go to a doctor for, you know, you have some 714 00:41:34,200 --> 00:41:37,400 medical problem and you know, you walk into the office, you 715 00:41:37,400 --> 00:41:40,760 have 20 minutes doctor sitting there next to a computer and is 716 00:41:40,760 --> 00:41:45,200 asking you questions, you know, about your symptoms, about your 717 00:41:45,200 --> 00:41:46,960 previous medical history. What is he doing? 718 00:41:46,960 --> 00:41:49,760 He's looking at the computer he's typing in because you got 719 00:41:49,760 --> 00:41:51,720 to get all that in the computer because you have to have a 720 00:41:51,720 --> 00:41:53,800 record, right? And then, you know, he's not 721 00:41:53,800 --> 00:41:56,320 looking at you. And then at the end, you know, 722 00:41:56,320 --> 00:41:59,240 he, he, he, he gives you a list of, of drugs that you should 723 00:41:59,240 --> 00:42:01,080 take or some advice and so forth. 724 00:42:01,080 --> 00:42:04,240 And, and off you go. And you know, you, maybe you 725 00:42:04,240 --> 00:42:06,640 get, you remember half of it if you're lucky, right? 726 00:42:07,080 --> 00:42:08,960 So here's what's happening right now. 727 00:42:08,960 --> 00:42:12,120 There are 10 companies out there that have realized I could put 728 00:42:12,120 --> 00:42:15,400 together a lot of different AI systems to make that process of 729 00:42:15,400 --> 00:42:20,160 not just a lot more efficient, but actually much more likely to 730 00:42:20,160 --> 00:42:22,360 be of help to the to the patient. 731 00:42:23,200 --> 00:42:26,000 So first of all, speech recognition, the doctor doesn't 732 00:42:26,000 --> 00:42:28,600 have to look at the computer. He can talk to the patient. 733 00:42:28,640 --> 00:42:30,680 It turns out you get a lot of information by looking at the 734 00:42:30,680 --> 00:42:32,800 patient, not just, you know, what they're saying. 735 00:42:33,200 --> 00:42:35,400 You can, you can you can tell from their facial expressions, 736 00:42:35,400 --> 00:42:36,800 you know, how serious something is. 737 00:42:36,800 --> 00:42:40,560 You can look at their face, you know the color of the face. 738 00:42:40,560 --> 00:42:44,480 I mean, a good diagnostician use can use that information right 739 00:42:44,480 --> 00:42:48,160 and, and jump to, you know, to conclusions that you can't just 740 00:42:48,160 --> 00:42:51,000 with a couple of numbers that they are that that you're 741 00:42:51,000 --> 00:42:55,160 getting, you know, typing in and OK, now what happens? 742 00:42:55,160 --> 00:43:00,400 So you press a button at the end and much more human interaction 743 00:43:00,640 --> 00:43:03,480 and now chat GDP does the heavy lifting. 744 00:43:03,480 --> 00:43:06,520 It gives you a summary. It's very good at that doctor 745 00:43:06,520 --> 00:43:09,560 looks at it and checks to see what's you know, if there's any 746 00:43:09,560 --> 00:43:12,080 problem in terms of the recommendations and so forth. 747 00:43:12,560 --> 00:43:16,560 And now you know that goes back and, and the the patient now 748 00:43:16,560 --> 00:43:19,200 walks out with actually very detailed summary. 749 00:43:19,200 --> 00:43:21,960 And so now the patient is going to be able to understand much 750 00:43:21,960 --> 00:43:24,920 better what needs to be done over the next couple of months, 751 00:43:24,920 --> 00:43:28,800 you know, in order for the patient to be able to, you know, 752 00:43:29,120 --> 00:43:32,800 improve their, their health. And, and you know, this, this is 753 00:43:32,800 --> 00:43:36,040 just you kind of one example of how we have a system right now, 754 00:43:36,040 --> 00:43:40,400 which is really disadvantages patients, right? 755 00:43:40,960 --> 00:43:44,280 And doctors, by the way, doctors don't just stop because they 756 00:43:44,280 --> 00:43:46,520 have to write up notes afterwards and often late into 757 00:43:46,520 --> 00:43:48,920 the night. And now all of that is being 758 00:43:48,920 --> 00:43:50,120 done. They could, they can go and 759 00:43:50,120 --> 00:43:52,680 they, you know, this is something that it's going to 760 00:43:52,680 --> 00:43:55,080 take decades to permeate the medical system. 761 00:43:55,080 --> 00:43:56,200 You know, they're very conservative. 762 00:43:56,200 --> 00:43:58,800 So it's going to take a long time and it has it'll have flaws 763 00:43:58,800 --> 00:44:02,080 and it'll there'll be problems, but ultimately I think that 764 00:44:02,080 --> 00:44:03,560 it'll greatly improve healthcare. 765 00:44:03,960 --> 00:44:08,680 Chris Peterson has a question, and very quickly please how does 766 00:44:08,680 --> 00:44:13,640 the enormous cost of retraining the big models play into GDPR 767 00:44:13,640 --> 00:44:17,360 style right to be forgotten laws? 768 00:44:17,600 --> 00:44:21,360 Will AI systems need an exception because it's just not 769 00:44:21,360 --> 00:44:25,600 practical to retrain for each request and very quickly? 770 00:44:25,840 --> 00:44:28,480 In addition to the initial training, which is the costly 771 00:44:28,480 --> 00:44:31,920 part, that is also opportunities later to do something called 772 00:44:31,920 --> 00:44:33,240 fine tuning. And what's that mean? 773 00:44:33,800 --> 00:44:41,440 Fine tuning is basically giving it extra data and trying to 774 00:44:41,440 --> 00:44:46,560 train it in a, in a gentle way such that it now has access to 775 00:44:46,560 --> 00:44:50,600 new information without interfering with the existing 776 00:44:50,600 --> 00:44:52,760 database. Now, it, it's not perfect. 777 00:44:52,760 --> 00:44:55,840 What happens is that the, the, the large language model gets 778 00:44:55,840 --> 00:44:58,360 dumbed down to some extent because it's taking on this 779 00:44:58,360 --> 00:45:01,000 extra burden. But that's not nearly as 780 00:45:01,000 --> 00:45:03,080 expensive. And that can be done in house, 781 00:45:03,080 --> 00:45:04,760 right? That could be done in individual 782 00:45:04,760 --> 00:45:06,920 businesses. You, you have to hire somebody 783 00:45:06,920 --> 00:45:10,480 who, who can do that for you. You know, that these are machine 784 00:45:10,480 --> 00:45:13,480 learning people. And by the way, the, the, you 785 00:45:13,480 --> 00:45:16,960 know, I, when I started, we were doing neural networks, right? 786 00:45:16,960 --> 00:45:20,000 And that was in the 80s. And then it'd be overtime 787 00:45:20,360 --> 00:45:22,760 Neurips became a machine learning conference. 788 00:45:23,080 --> 00:45:26,000 But now it, it, it's come back, you know, full circle. 789 00:45:26,400 --> 00:45:28,880 And now it's, it's not called neural networks anywhere. 790 00:45:28,880 --> 00:45:32,840 It's called AI, right? You know, and it wasn't because 791 00:45:32,840 --> 00:45:34,560 we called it that. It was because the world was 792 00:45:34,560 --> 00:45:36,840 calling it that. But it's really about machine 793 00:45:36,840 --> 00:45:39,000 learning. That's the heart of AI today. 794 00:45:39,280 --> 00:45:44,320 And it's all about data. Whoever has more data wins in 795 00:45:44,320 --> 00:45:47,000 any area. And I'll tell you your company 796 00:45:47,000 --> 00:45:50,600 is sitting on enormous amounts of data that's really important 797 00:45:50,600 --> 00:45:53,120 for you. And if you can get that into an 798 00:45:53,160 --> 00:45:56,960 AI, you will win. Can you talk about how the 799 00:45:56,960 --> 00:46:03,040 assumptions made during data curation influence AI outcomes 800 00:46:03,440 --> 00:46:07,040 usually? This is really going to become a 801 00:46:07,040 --> 00:46:13,800 very large business data, as I said earlier, data is hard to 802 00:46:13,800 --> 00:46:16,840 come by quality data even harder. 803 00:46:18,600 --> 00:46:26,400 And and so, you know, how do, how do you, how, how is it that 804 00:46:26,400 --> 00:46:30,840 we're going to be able to overcome a lot of the problems? 805 00:46:33,000 --> 00:46:35,400 You know, hallucination is a good example. 806 00:46:37,440 --> 00:46:42,880 You know, hallucination actually is, is has has advantages. 807 00:46:42,880 --> 00:46:46,440 If you're a writer or, you know, ad copy, it's really good at 808 00:46:46,440 --> 00:46:50,720 that or poems, but not so good if you're asking for a fact 809 00:46:52,080 --> 00:46:54,800 hallucinates, you're in trouble. So I think what we're going to, 810 00:46:54,800 --> 00:46:57,480 we're going to need is another layer on top of the, of the 811 00:46:57,480 --> 00:47:01,080 language models, which is, is, is a fact checker basically. 812 00:47:01,080 --> 00:47:05,520 And, and will, will take what is coming out and we'll learn how 813 00:47:05,520 --> 00:47:08,760 to then look through the database world's database and, 814 00:47:08,760 --> 00:47:13,480 and come up with, you know, sources for that particular 815 00:47:13,480 --> 00:47:18,040 fact. So again, it may be this isn't 816 00:47:18,040 --> 00:47:20,600 really the exciting thing is that right now humans are doing 817 00:47:20,600 --> 00:47:21,720 that. You know, that all these big 818 00:47:21,720 --> 00:47:24,000 companies have thousands and thousands of humans that are 819 00:47:24,000 --> 00:47:27,880 going through and sorting through getting rid of racist 820 00:47:28,560 --> 00:47:33,280 data, trying to, you know, make prevent of these large language 821 00:47:33,280 --> 00:47:35,800 models from saying from doing anything that is going to hurt 822 00:47:35,800 --> 00:47:37,840 people. You know, that that is being 823 00:47:37,840 --> 00:47:41,240 done on a individual basis. And that's very labor intensive. 824 00:47:41,720 --> 00:47:46,040 But you know, we, we should be able to train another generation 825 00:47:46,040 --> 00:47:49,480 of large language models to actually do that part of the job 826 00:47:49,480 --> 00:47:53,160 of, of, of, of sorting through and flagging things. 827 00:47:53,440 --> 00:47:56,880 And, and, you know, even in the earliest days of, for example, 828 00:47:56,880 --> 00:48:00,160 neural networks in the 80s, when I started, they were using these 829 00:48:00,400 --> 00:48:02,880 with very, very, at that time, they were very shallow with one 830 00:48:02,880 --> 00:48:08,960 layer of pin units to take a slide with cancerous, it was 831 00:48:08,960 --> 00:48:12,040 called a Pap smear, you know, the cervical cancer and, and 832 00:48:12,200 --> 00:48:14,240 humans would have to look through the every cell and it 833 00:48:14,240 --> 00:48:16,280 would take hours and hours and very costly. 834 00:48:16,520 --> 00:48:19,360 Well, you know, neural networks could do that much more quickly, 835 00:48:19,360 --> 00:48:22,600 reduce it to 100. And then the humans were going 836 00:48:22,600 --> 00:48:27,160 to, you know, use their much more powerful visual system to 837 00:48:27,200 --> 00:48:30,640 look at the, the debris and, and, and all the different types 838 00:48:30,640 --> 00:48:36,080 of cells and, and, and come up with a much actually thorough 839 00:48:36,080 --> 00:48:38,520 and faster way of analyzing the data. 840 00:48:38,880 --> 00:48:42,800 And so the idea is we have layers and, and humans are one 841 00:48:42,800 --> 00:48:44,320 of the layers. They could be the layer at the 842 00:48:44,320 --> 00:48:46,720 top or they could be in the middle or, you know, anywhere. 843 00:48:46,880 --> 00:48:49,720 But the idea though, is that we have different systems that are 844 00:48:49,720 --> 00:48:52,960 there for spotting different problems in the data. 845 00:48:53,640 --> 00:48:55,960 And, and this is, again, this is where we're at the very 846 00:48:55,960 --> 00:48:57,960 beginning of this, this is going to take decades. 847 00:48:59,040 --> 00:49:01,000 You know, it, it, it took it literally. 848 00:49:01,000 --> 00:49:02,640 Just think of the Wright brothers, right? 849 00:49:02,720 --> 00:49:05,320 Look how long it took to get to jet engines, right? 850 00:49:05,320 --> 00:49:09,120 And and jet planes now you know you can go across the world now, 851 00:49:09,120 --> 00:49:12,600 but back then, right, you could be be lucky to just go across 852 00:49:13,040 --> 00:49:17,160 the countryside. So, on the subject of humans and 853 00:49:17,160 --> 00:49:22,880 datas and data, how can we detect and understand the impact 854 00:49:22,880 --> 00:49:27,000 of adversaries manipulating data, for example, with 855 00:49:27,000 --> 00:49:29,600 misinformation and disinformation? 856 00:49:30,040 --> 00:49:32,800 Well, that's already happening. You know, we're social media. 857 00:49:32,800 --> 00:49:36,160 This is filled with that, you know, that that's, that's a, a 858 00:49:36,160 --> 00:49:39,240 very, very difficult problem. I don't think it's going to be 859 00:49:39,240 --> 00:49:42,680 any easy solution. I, I outlined just now what I 860 00:49:42,680 --> 00:49:45,640 think is going to happen with all these layers of, of, of, of 861 00:49:45,640 --> 00:49:51,080 filtering. But you know, there's, I think 862 00:49:51,520 --> 00:49:56,160 one thing that we should be open to is the fact that every once 863 00:49:56,160 --> 00:49:58,960 in a while there's a, a massive a breakthrough. 864 00:49:58,960 --> 00:50:05,040 The last one occurred in 2022 when chat GDP was opened up to 865 00:50:05,040 --> 00:50:08,200 the world and suddenly everybody realized, oh, my God. 866 00:50:09,600 --> 00:50:11,320 Yeah. What hath God wrought? 867 00:50:12,000 --> 00:50:14,880 That was the that was the first message sent over the Telegraph. 868 00:50:15,000 --> 00:50:20,200 And we're going to have it. I'm sure we'll have another 869 00:50:20,200 --> 00:50:22,640 moment like that. I don't know when, I don't know 870 00:50:22,640 --> 00:50:24,960 how, but you know, it's going to happen. 871 00:50:24,960 --> 00:50:27,760 It's going to happen, and it's already happened. 872 00:50:27,760 --> 00:50:30,280 And I told you in science, it's happened multiple times. 873 00:50:30,280 --> 00:50:33,680 I'll just give you one example. Protein folding, how does it a 874 00:50:33,680 --> 00:50:36,600 string of amino acids get folded up to become an enzyme that has 875 00:50:36,600 --> 00:50:39,800 a function and that turns out to be a computationally 876 00:50:40,040 --> 00:50:41,480 intractable. You can't do it. 877 00:50:41,480 --> 00:50:44,200 It just takes too much for computing power and, and most 878 00:50:44,200 --> 00:50:45,800 biologists thought it would never be solved, but we've 879 00:50:45,800 --> 00:50:49,280 solved it. Alpha fold DeepMind created a 880 00:50:49,280 --> 00:50:52,080 transformer that could solve that problem and and it's 881 00:50:52,080 --> 00:50:55,520 transformed biology. This is like a huge why because 882 00:50:55,520 --> 00:50:59,960 you can now design enzymes that have much more specific actions 883 00:51:00,240 --> 00:51:04,760 and better drugs, better ways of, of being able to make 884 00:51:04,760 --> 00:51:08,800 predictions about how different interactions are going to occur 885 00:51:08,800 --> 00:51:12,160 between proteins. And this is really a 886 00:51:12,160 --> 00:51:16,640 transformation in the whole business of, of creating better 887 00:51:16,640 --> 00:51:19,920 healthcare, better drugs. So it's going to happen in many, 888 00:51:19,920 --> 00:51:22,520 many areas in ways you can't predict. 889 00:51:22,840 --> 00:51:27,880 What advice do you have for business people given everything 890 00:51:27,880 --> 00:51:33,320 we've discussed? Number one, read my book, not my 891 00:51:33,320 --> 00:51:35,160 lips. Read the book. 892 00:51:36,200 --> 00:51:38,720 Because I talk a lot about these issues that, I mean, I don't 893 00:51:38,720 --> 00:51:40,600 really touch the surface of what is in the book. 894 00:51:40,600 --> 00:51:44,280 OK #2 the most important thing you should be thinking about is 895 00:51:44,280 --> 00:51:47,560 training your workforce with this new technology and take it 896 00:51:47,560 --> 00:51:50,240 seriously. It's, it's going to, it's it, 897 00:51:50,240 --> 00:51:51,840 this is just the beginning, right? 898 00:51:51,840 --> 00:51:55,400 And, and unless you start now, right, helping your workforce, 899 00:51:55,840 --> 00:51:59,520 we use these tools, the new tools and, and it really is that 900 00:51:59,520 --> 00:52:02,640 they're tools that if you can use them badly and, and, and 901 00:52:02,640 --> 00:52:04,760 very well. And you know, people are trained 902 00:52:04,760 --> 00:52:06,120 in that. And it does take time. 903 00:52:06,120 --> 00:52:08,000 It takes, you know, maybe 100 hours. 904 00:52:10,160 --> 00:52:12,600 That's a lot less than a 10,000 hours it takes to become an 905 00:52:12,600 --> 00:52:16,400 expert, right? And #3 download your company 906 00:52:16,400 --> 00:52:21,360 into ALLM. And I'll tell you, this is 907 00:52:21,360 --> 00:52:26,560 already being done with the brains of flies and zebrafish. 908 00:52:26,560 --> 00:52:28,640 These are brains that have 100,000 neurons. 909 00:52:28,640 --> 00:52:32,760 We can download them now into AI and and they have similar 910 00:52:33,160 --> 00:52:35,960 behaviors, they have similar activity patterns. 911 00:52:36,240 --> 00:52:40,520 You know that's the future. My mind is blown from this 912 00:52:40,520 --> 00:52:43,920 conversation. Terry Signowski, thank you so 913 00:52:43,920 --> 00:52:46,480 much for taking time to be with us. 914 00:52:46,880 --> 00:52:49,960 Oh, I'm, I'm very pleased. Thank you so much for making 915 00:52:49,960 --> 00:52:51,320 this opportunity possible for me. 916 00:52:51,840 --> 00:52:55,440 And a huge thank you to everybody who watched. 917 00:52:55,440 --> 00:52:58,160 You guys are amazing. You asked such excellent 918 00:52:58,160 --> 00:53:02,200 questions. Before you go, please subscribe 919 00:53:02,200 --> 00:53:05,920 to our newsletter, go to cxotalk.com, subscribe to our 920 00:53:05,920 --> 00:53:10,720 newsletter, subscribe to our YouTube channel, and we have 921 00:53:10,720 --> 00:53:13,960 amazing shows coming up. Thank you so much everybody, and 922 00:53:13,960 --> 00:53:15,800 I hope you have a great day. Take care.