1
00:00:00,040 --> 00:00:05,640
What would it look like if if 
AGI or AI gets so good we trust 

2
00:00:05,640 --> 00:00:08,960
it to manage something like the 
world economy? 

3
00:00:08,960 --> 00:00:12,320
What does that look like? 
Well, hopefully it looks like a 

4
00:00:12,320 --> 00:00:15,800
very well functioning world 
economy, but that can also be a 

5
00:00:15,800 --> 00:00:18,640
bit of a problem. 
So why do we have legal systems 

6
00:00:18,640 --> 00:00:21,560
and States and this? 
Well, that is the way we 

7
00:00:21,560 --> 00:00:25,040
outsource our cognition and we 
handle it by having these 

8
00:00:25,120 --> 00:00:28,000
artificial systems made out of 
people and pieces of paper. 

9
00:00:28,680 --> 00:00:31,320
But now we're of course also 
getting software that can take 

10
00:00:31,320 --> 00:00:34,000
over more and more of that. 
Why do we have law? 

11
00:00:34,040 --> 00:00:37,000
Well, we want peace and 
prosperity and of course, power 

12
00:00:37,000 --> 00:00:39,320
to some people. 
They are very good at latching 

13
00:00:39,320 --> 00:00:41,720
onto that. 
And with AI, we might get even 

14
00:00:41,720 --> 00:00:44,720
more peace and prosperity, even 
more efficient, and maybe power 

15
00:00:44,720 --> 00:00:48,200
to some old man or other people 
who happen to get grasp it in 

16
00:00:48,200 --> 00:00:50,440
the right way. 
But that's a separate issue. 

17
00:00:51,280 --> 00:00:54,240
But the really interesting part 
is we might really want better 

18
00:00:54,240 --> 00:00:57,440
coordination because the world 
is big and somewhat dangerous. 

19
00:00:57,440 --> 00:01:00,920
We have climate change, We have 
controlling, powerful AI. 

20
00:01:00,920 --> 00:01:03,760
We have biotechnology getting 
more and more powerful. 

21
00:01:04,120 --> 00:01:07,280
We probably need to coordinate 
about some global problems way 

22
00:01:07,280 --> 00:01:10,520
more than powerfully than we can
do right now. 

23
00:01:10,880 --> 00:01:14,000
And maybe we could do that with 
the United Nations Plus Plus, 

24
00:01:14,440 --> 00:01:17,320
but that seems a bit unlikely. 
It might be that we actually 

25
00:01:17,480 --> 00:01:20,680
want to leave more and more to 
super smart distributed AI 

26
00:01:20,680 --> 00:01:23,120
systems. 
But then you end up with a funny

27
00:01:23,120 --> 00:01:25,440
situation. 
We might be swimming in peace 

28
00:01:25,440 --> 00:01:26,960
and prosperity, and that's 
great. 

29
00:01:27,520 --> 00:01:30,040
But previously we needed human 
autonomy. 

30
00:01:30,040 --> 00:01:33,360
We need to make decisions to run
the legal system, to run the 

31
00:01:33,360 --> 00:01:36,280
market. 
Now it runs itself and probably 

32
00:01:36,280 --> 00:01:38,960
runs itself much better without 
us messing with it. 

33
00:01:40,080 --> 00:01:41,760
It's a little bit like chess 
playing. 

34
00:01:42,240 --> 00:01:45,120
Once humans were better than 
computers are playing chess. 

35
00:01:45,320 --> 00:01:47,760
Then gradually computers got 
better and better and started 

36
00:01:47,760 --> 00:01:50,600
beating humans. 
But the team of humans and 

37
00:01:50,600 --> 00:01:53,040
computers could actually still 
do better. 

38
00:01:53,240 --> 00:01:55,880
But after a while, the computer 
was so good that actually the 

39
00:01:55,880 --> 00:01:57,440
humans were you're slowing 
things down. 

40
00:01:57,640 --> 00:01:59,840
So actually you should just 
listen to the computer. 

41
00:02:00,480 --> 00:02:03,720
So if we end up in that world, 
we have this weird situation 

42
00:02:03,720 --> 00:02:07,280
that we might be like pampered 
aristocrats, really good 

43
00:02:07,280 --> 00:02:10,840
material standard, but in some 
important respects our lives are

44
00:02:10,840 --> 00:02:14,000
not our own. 
And I think that sounds bad to 

45
00:02:14,000 --> 00:02:16,800
many people. 
And I think the reason is we 

46
00:02:16,800 --> 00:02:19,920
want to feel like we're in 
control because all our 

47
00:02:19,920 --> 00:02:23,040
ancestors, the ones that were 
kind of find giving up control 

48
00:02:23,040 --> 00:02:26,120
to others, they were just 
exploited by the ruthless 

49
00:02:26,120 --> 00:02:28,360
people. 
So we have this Ted, this event,

50
00:02:28,360 --> 00:02:32,200
we want to be have a tournament.
And then The funny thing is once

51
00:02:32,200 --> 00:02:35,040
we have autonomy, they're very, 
we are very happy to give it 

52
00:02:35,040 --> 00:02:38,520
away in various ways, but we 
want to be the ones that do it. 

53
00:02:38,760 --> 00:02:41,200
It might be that you give away 
your autonomy by getting 

54
00:02:41,200 --> 00:02:42,920
married. 
It might be that you join the 

55
00:02:42,920 --> 00:02:47,680
company and in the exchange for 
getting a paycheck you get some 

56
00:02:47,680 --> 00:02:51,360
limitation of what you can do, 
but that might be worth it, so I

57
00:02:51,400 --> 00:02:53,840
do. 
Yep, kids are amazing. 

58
00:02:53,840 --> 00:02:57,640
Except people. 
Every person I know that has 

59
00:02:57,640 --> 00:03:01,080
gotten kids say, Oh, yes, it's 
super stressful and there's a 

60
00:03:01,080 --> 00:03:04,280
lot of hard work, but it's the 
most important meaningful thing 

61
00:03:04,280 --> 00:03:06,920
I've ever done. 
And it's amazing. 

62
00:03:06,920 --> 00:03:10,320
It's almost like mind control as
somebody who haven't got a kid, 

63
00:03:10,320 --> 00:03:13,360
but it's kind of a happy uncle. 
It's kind of, whoa, those 

64
00:03:13,360 --> 00:03:16,040
babies, they, I really got some 
psionic power here. 

65
00:03:16,720 --> 00:03:19,840
But it's very effective. 
And it's kind of good for our 

66
00:03:19,840 --> 00:03:22,760
species that we have it. 
And it's good that we can give 

67
00:03:22,760 --> 00:03:24,960
away autonomy in a reasonable 
way. 

68
00:03:25,160 --> 00:03:27,440
But we should also in some sense
be able to withdraw it. 

69
00:03:27,720 --> 00:03:30,440
Although, of course, when you're
a parent, you realize that, OK, 

70
00:03:30,440 --> 00:03:32,280
I cannot even withdraw a bad 
thing. 

71
00:03:32,480 --> 00:03:35,440
Now I'm in for it because it's 
what it is. 

72
00:03:36,320 --> 00:03:39,640
But this is where a world 
economy run by AI might be 

73
00:03:39,640 --> 00:03:42,960
tricky because it might be so 
effective that, yeah, running 

74
00:03:42,960 --> 00:03:45,880
your own park might be in out 
compete. 

75
00:03:45,880 --> 00:03:48,280
That might be so inefficient. 
It's a bit like living your life

76
00:03:48,280 --> 00:03:52,360
without electricity and a little
log cabin in the woods sounds 

77
00:03:52,360 --> 00:03:55,640
absolutely lovely, and probably 
totally lovely the first day 

78
00:03:56,080 --> 00:03:59,520
after a few weeks of carrying 
water, a bit less lovely. 

79
00:03:59,760 --> 00:04:03,000
And after a month, most people 
are back in the writing on 

80
00:04:03,000 --> 00:04:06,240
Facebook about experience. 
We can have two economies, like 

81
00:04:06,240 --> 00:04:08,440
we'll have two of me and two of 
you and two of everybody else. 

82
00:04:08,440 --> 00:04:11,320
You have two economies, one 
that's run efficiently and well 

83
00:04:11,320 --> 00:04:14,280
by the AGI, and then another 
smaller one that's run and 

84
00:04:14,280 --> 00:04:18,480
managed by humans and they just 
run in conjunction with each 

85
00:04:18,480 --> 00:04:19,760
other. 
Yeah. 

86
00:04:19,880 --> 00:04:22,320
And in some sense, I think that 
this might be what we're 

87
00:04:22,320 --> 00:04:25,160
developing. 
We have the backbone of our 

88
00:04:25,160 --> 00:04:27,480
civilization, the big 
infrastructure, the really 

89
00:04:27,480 --> 00:04:30,080
important things that we must 
get right. 

90
00:04:30,640 --> 00:04:33,560
When the electricity goes well, 
nothing else works. 

91
00:04:33,560 --> 00:04:35,320
So you really need to get that 
working. 

92
00:04:35,640 --> 00:04:38,680
But once you're wealth enough, 
you can spend that excess wealth

93
00:04:38,960 --> 00:04:41,840
on all sorts of weird stuff. 
There is a reason why you have 

94
00:04:41,840 --> 00:04:43,800
so many consultants as big 
companies. 

95
00:04:43,800 --> 00:04:46,440
They can't afford it and they're
probably more involved with 

96
00:04:46,440 --> 00:04:50,240
status games and politics than 
actually doing something useful.

97
00:04:50,720 --> 00:04:54,280
I think it's a good thing, but 
we have enough wealth and the 

98
00:04:54,280 --> 00:04:57,200
safety that we can try out 
different lifestyles, even if 

99
00:04:57,280 --> 00:04:59,720
many of them turn out to be a 
slightly embarrassing once 

100
00:04:59,720 --> 00:05:02,080
you're middle-aged. 
There are things as humans, 

101
00:05:02,520 --> 00:05:08,320
power dynamics and control and 
me versus you, like how if we 

102
00:05:08,320 --> 00:05:11,840
were to design or not design, 
but like kind of enable this 

103
00:05:12,200 --> 00:05:17,480
world economy through AI. 
How do we make sure as humans we

104
00:05:17,480 --> 00:05:20,240
don't bring that baggage, that. 
Always wrecks. 

105
00:05:20,320 --> 00:05:23,600
Up the human but. 
Even that is a problem when you 

106
00:05:23,600 --> 00:05:26,960
try to remove the human. 
Well, what have you trained the 

107
00:05:26,960 --> 00:05:29,920
large language models on? 
They're all trained on all the 

108
00:05:29,920 --> 00:05:32,800
human literature. 
They have read every single 

109
00:05:32,800 --> 00:05:35,960
thread we've written about 
betrayal and power dynamics and 

110
00:05:35,960 --> 00:05:39,880
corporate shenanigans. 
One of the big debates right now

111
00:05:39,880 --> 00:05:44,280
in the AI safety communities 
when you test your AI models and

112
00:05:44,280 --> 00:05:47,880
they try to do sneaky stuff like
escape the system, if they think

113
00:05:47,880 --> 00:05:51,440
of it's going to get shut down, 
is that because they want in 

114
00:05:51,440 --> 00:05:54,480
some sense to escape or? 
Yes, because they know that's 

115
00:05:54,480 --> 00:05:57,560
expect of me because of all the 
science fiction I ever read says

116
00:05:57,560 --> 00:06:00,840
that that's what an AI does in 
the situation and maybe it 

117
00:06:00,840 --> 00:06:04,040
doesn't even matter. 
So I do think that it's very 

118
00:06:04,040 --> 00:06:08,240
hard now to take away the 
imprint of the human world from 

119
00:06:08,280 --> 00:06:10,920
the AI world on that round. 
I think it is also going to be 

120
00:06:10,920 --> 00:06:13,760
evolutionary pressures pushing 
in certain directions. 

121
00:06:14,240 --> 00:06:16,000
There are various forms of 
competition. 

122
00:06:16,000 --> 00:06:19,040
That means that certain soft 
pieces of software are going to 

123
00:06:19,040 --> 00:06:23,360
be copied more widely than ours,
and that is going to act anyway 

124
00:06:23,720 --> 00:06:25,440
regardless what human values 
are. 

125
00:06:25,720 --> 00:06:28,040
And of course, my more do me 
friends say yeah. 

126
00:06:28,040 --> 00:06:31,800
And that will inevitably push in
a very scary direction that's 

127
00:06:31,800 --> 00:06:34,480
going to be very different from 
what we humans are. 

128
00:06:35,080 --> 00:06:37,840
So I think one important thing 
we might want to be involved 

129
00:06:37,840 --> 00:06:41,360
here, we might want to be an 
essential spice in this mix to 

130
00:06:41,360 --> 00:06:44,000
make sure that there is at least
some of us in that. 

131
00:06:44,440 --> 00:06:48,000
But then we want to filter out 
the really bad stuff, Cure, cure

132
00:06:48,480 --> 00:06:51,240
cruelty. 
For example, it's kind of useful

133
00:06:51,240 --> 00:06:53,400
to stabilize altruistic 
punishment. 

134
00:06:54,240 --> 00:06:57,920
It's taking the light in the 
misfortune of somebody else is a

135
00:06:57,920 --> 00:07:01,360
really good way of kind of 
giving you extra points when 

136
00:07:01,360 --> 00:07:03,440
you're trying to punish somebody
breaking norms. 

137
00:07:03,640 --> 00:07:07,240
Because otherwise I would be 
taking a risk in the policing 

138
00:07:07,240 --> 00:07:09,280
these norms. 
And actually it would be easier 

139
00:07:09,280 --> 00:07:11,120
if I stayed at home and let the 
norm break. 

140
00:07:11,120 --> 00:07:14,400
You get away with it, except 
that today we can do this with 

141
00:07:14,400 --> 00:07:16,600
laws. 
We can use police we and 

142
00:07:16,600 --> 00:07:19,520
suddenly cruelty is just a bad 
thing that generally makes us 

143
00:07:19,520 --> 00:07:22,040
nasty. 
That's an emotion I totally 

144
00:07:22,040 --> 00:07:25,200
think we should kind of try to 
get rid of and at the very least

145
00:07:25,200 --> 00:07:27,520
make sure our AI progress. 
Never experience.