AI Takeover is Inevitable – Richard Sutton

you have a perspective that's quite different from a lot of people that I've interviewed and maybe a lot of people generally. So I also think it's a very

generally. So I also think it's a very interesting perspective. I want to hear

interesting perspective. I want to hear about it.

>> Yeah. So I do think succession to digital or digital intelligence or augmented humans is inevitable. So the argument go

I have a four-part argument. Now your

step one is there's no government or organization that that uh gives humanity a unified point of view that dominates and that

can that can arrange. There's no

consensus about how the world should be run. And number two um we will figure

run. And number two um we will figure out how intelligence works. Researchers

will figure it out eventually. And

number three we won't stop just with human level intelligence. we will get reach super intelligence. And number

four is that once it's inevitable over time that the most intelligent things around would gain resources and power.

Uh and uh so put all that together, it's you know you um it's sort of inevitable that you're going to have um succession

to AI or to AI enabled augmented humans.

So within those those four things seem clear and and and sure to happen. Uh but

within that set of possibilities some there could be good outcomes as well as as less good outcomes bad outcomes >> and um so I just just trying to be realistic

about where we are and and ask how we should feel about it. Yeah, I I agree with all four of those arguments and the implication and I also agree that

succession contains a wide variety of possible futures. So

curious to get more thoughts on that.

>> Right. And so then I do encourage people to think um positively about it first of all because it's something we humans have always tried to do for thousands of years. try to understand themselves,

years. try to understand themselves, trying to make themselves think better and um you know just understand themselves. So

this is a great success from as science humanities uh we're finding out what this essential part of of of humanness

is what it means to be intelligent. And

then what I usually say is is that this is all kind of humanentric. What if we look we step aside from being a human and just say take the point of view of the universe and and this is I think a

major stage in the universe a major transition a transition from replicators we humans and animals plants we're all replicators and that gives us some

strengths and some limitations and then we're entering the age of design where because our AIs are designed our our our all of our physical objects are designed our buildings are designed our

technology is designed and we're we're designing now uh AIs things that can be intelligent themselves and that are themselves capable of design and so this

is this is a key step in the world and I and in the universe and I think it's the it's the transition from the world in which most of the interesting things

uh that are are replicated replicated means you can make copies of them uh but you don't really understand them like right now we make more intelligent beings, more children. Uh but we don't really understand how intelligence

works.

>> Whereas in as we're we're reaching now to having designed intelligence, intelligence that we do understand how it works and therefore we can change it in different ways in a different speeds

um than otherwise and and our future they might not be replicated at all.

like we may just design AIs and those AIs will design other AIs and um everything will be done by design construction rather than by replication.

>> Yeah, I mark this as one of the four great stages of the universe. First

there's there's dust ends with stars.

Stars we and and then stars make planets and the planets give rise to life and now we're giving life life to uh designed entities.

If you enjoyed this clip, you can watch the full episode here and subscribe for more clips. Thanks.

more clips. Thanks.