《人類大歷史》作者解釋 AI 如何毁滅人類 - Yuval Noah Harari 哈拉瑞 |《連結 Nexus》作者

the algorithms discovered the easiest way to grab people's attention and keep them glued to the screen is by pressing the greed or hate or fear button in our

minds the people who manage the social media companies they are not evil they just really didn't foresee this is the problem I mean because we don't know if they really have Consciousness or

they're only very very good at mimicking Consciousness um and in the Hollywood scenario you have the Killer Robots in people in real life it's the humans

pulling the trigger but the AI is choosing the [Music] targets I think maybe the most important thing is is really to understand what

what AI is because now there is so much hype around AI that it's becoming difficult for people to understand what is AI now everything is AI so you know

your coffee machine is now a coffee machine is an AI coffee machine and your shoes are AI shoes and what is AI you know the key thing to understand is that

AIS are able to learn and change by themselves to make decisions by themselves to invent new ideas by themselves if a machine cannot do that

it's not really an AI and therefore by definition something that we cannot predict how it will develop and evolve

and for good or or for bad it can invent medicines and treatments we never thought about but it can also invent weapons and dangerous strategies that go

beyond our imagination you characterize AI not as artificial intelligence but as alien intelligence you give it a different term can you explain the

difference there and why you why you've landed on that word yeah traditionally the acon AI stood for artificial

intelligence but with every passing year AI becomes less artificial and more alien alien not in the sense that it's coming from out of space it's not uh we

create it but alien in the sense it analyzes information makes decisions invents new things in a fundamentally different way than human beings MH again

artificial is from artifact it give us the impression that this is an artifact that we control and this is misleading because yes we designed the kind of baby

AI we we gave them the ability to learn and change by themselves and then we release them to the world and they do things that are not under our control

that are unpredictable so in this sense they are alien not in the sense that they came from ours when I said earlier that you know AI can make decisions and

AIS they are not just Tools in our hands they are agents creating new realities so you may think okay this is a prophecy for the future a prediction about the

future but it's already in the past because even though social media algorithms they are very very primitive AIS you know the fair generation of AIS

they still reshaped the world with the decisions they made in social media on Facebook Twitter Tik Tok all that the ones that make the decision what you

will see at the top of your news feed or the next video that you'll be recommended It's Not a Human Being sitting there making these decisions it's an AI it's an algorithm and these

algorithms were given a relatively simple and seemingly benign goal by the corporations the goal was increase user

engagement which means in simple English make people spend more time on the platform uh because the more time people spend on Tik Tok or Facebook or Twitter or whatever the company makes more money

it sells more advertisements it harvests more data that it can then sell to third parties so more time on the platform good for the company this is the goal of

the algorithm now engagement sounds like a good thing who doesn't want to be engaged but the algorithms then experimented on billions of human

guineapigs and discovered something which was of course discovered even earlier by humans but now the algorithms discovered it the algorithms discovered that the

easiest way to increase user engagement the easiest way to grab people's attention and keep them glued to the

screen is by pressing the greed or hate or fear button in our minds you show us some hatefi conspiracy theory and we

become very angry we want to to see more we tell about it to all our friends user engagement goes up and this is what they did over the last 10 or 15 years they

flooded the world with hate and greed and fear which is why again the conversation is breaking down these are kind of unintended consequences like the people who manage

the social media companies they are not evil they didn't set out to destroy democracy or to flood the world with with hate and and and so forth um they

just really didn't foresee that when they give the algorithm the goal of increasing us engagement the algorithm will start to promote hate but initially

when they started this whole ball rolling they really didn't know and this is just kind of a warning of look what happens with even very primitive AIS and

the AIS of today which are far more sophisticated than in 2016 they too are still just the very early stages of the AI evolutionary

process and we can think about it like the evolution of of animal until you get to humans you have 4 billion years of evolution you start

with microorganisms like amibas and it took billions of years of evolution to get to dinosaurs and mammals and humans

now AIS are present at the beginning of a parallel process the CH GPT and so forth they are the amibas of the AI

world but AI evolution is not organic it's inorganic it's digital and it's millions of times faster so what it took billions of years to get from amibas to

dinosaurs it might take just 10 or 20 years to get from the AI amibas of today to AI T-Rex consumers like all of us

we're we're being lured into a trust of something so powerful we can't comprehend and our IL equipped to be able to kind of cast our gaze into the

future and imagine where this is leading us absolutely I mean part of it is that there is enormous positive potential in AI it's not like it's all doom and gloom there is really enormous positive

potential if you think about the implications for healthcare that you know AI doctors available 24 hours a day that know our entire medical history and

have' read every medical paper that was ever published and can tailor their advice their treatment to our specific life history and our blood pressure our

gen gentics it it can be the biggest revolution in healthcare ever if you think about self-driving Vehicles so every year more than a million people die all over the world in car accidents

most of them are caused by human error like people drinking and then driving or falling asleep at the wheel or whatever uh cell driving vehicles are likely to save about a million lives every year

this is amazing you think about climate change so yes developing the AIS will consume a lot of energy but they could also find new sources of energy new ways

to to harness energy that could be our best shot at at preventing ecological collapse uh so there is enormous positive potential we shouldn't deny that we should be aware of it and on the

other hand it's very difficult to appreciate the dangers because the dangers again they are kind of alien like if you think about nuclear energy yeah it also had positive potential

nuclear cheap nuclear energy but people had a very good grasp of the danger nuclear war anybody can understand the danger of that with AI it's much more complex because

the danger is not straightforward the danger is really I mean we we've seen the Hollywood science fiction scenarios of the big robot Rebellion that one day

a big computer or the AI decides to take over the world and kill us or enslave us and this is extremely unlikely to happen anytime soon because the AIS are still a

kind of very narrow intelligence like the AI that can summarize a book it it doesn't know how to act in the physical world outside you have AIS that can fold proteins you have AIS that can play

chess but we don't have this kind of General AI that can just find its way around the world and build a robot army and and whatever so people it's hard to

understand so what's so dangerous about something which is so kind of narrow in its abilities and I would say that the danger doesn't come from the big robot Rebellion it comes from the AI

bureaucracies already today and more and more we will have not one big AI trying to take over the world we will have millions and billions of AIS constantly

making decisions about us everywhere you apply to a bank to get a loan it's an AI deciding whether to give you a loan you apply to get a job it's an AI deciding whether to give you a job you're in

court or you're found guilty of some crime the AI will decide whether you go for 6 months or 3 years or whatever even in armies we already see now in the war

in Gaza in the were in Ukraine AI make the decision about what to bomb um and in the Hollywood scenario you have the Killer Robots shooting people in real

life it's the humans pulling the trigger but the AI is choosing the targets I start thinking about like this this bias I have around the originality of human

thought and emotion and this kind of assumption that AI will never be able to fully mimic the Human Experience right there's something indelible about what

it means to be human that the machines uh will never be able to fully replicate and when you talk about you know information the purpose of information

being to create connection a big piece there is intimacy like intimacy between human beings so information is meant to create connection but now we have so much information and we're feeling very

disconnected so there's something broken in this system and I think it's driving this loneliness epidemic but on the other side it's it's making us value

like intimacy maybe a little bit more than we were previously uh and so I'm curious about where intimacy kind of fits into this you know posthuman World

in which culture is being dictated by machines I mean human beings are wired for that kind of intimacy and I think our radar or our kind of ability to you

know identify it when we see it is part of what makes us human to begin with maybe the most important part um I think the key distinction here that is often

lost is the distinction between intelligence and Consciousness that intelligence is the ability to pursue goals and to overcome

problems and obstacles on the way to the goal the goal could be a self-driving vehicle trying to get from here to San Francisco the goal could be increasing

user user engagement and an intelligent agent knows how to overcome the problems on the way to the goal this is intelligence and this is something that

AI is definitely acquiring in at least certain Fields AI is now much more intelligent than us like in playing chess much more intelligent than human

beings but Consciousness is a different thing than intelligence Consciousness is the ability to feel things pain pleasure love hate uh when the AI wins a game of

chess it's not joyful if there is a tense moment in the in the game it's not clear who is going to win the AI is not tense it's only the human player which

is tense or frightened or anxious the AI doesn't feel anything now there was a big confusion because in humans and also in

other mammals in other animals in dogs and pigs and horses and whatever intelligence and Consciousness go together we solve problems B based on

our feelings our feelings are not something that kind of evolution decoration it's the core system through which mammals make decisions and solve

problems is based on our feelings so we tend to think that Consciousness and intelligence must go together and in all these science fiction movies you see that as the computer or robot becomes

more intelligent then at some point it also gains Consciousness it falls in love with the the human or whatever and we have no reason to think

like that yeah Consciousness is not a mere extrapolation of intelligence a qualitatively different thing yeah and again if you think in terms of evolution

so yes the evolution of mammals took a certain path a certain Road in which you develop intelligence based on

Consciousness but so far what we see is computers they took a different route their Road develops intelligence without Consciousness I mean computers have been

developing you know for 60 70 years now they are not very intelligent at least in some fields and still zero Consciousness now this could continue indefinitely maybe they are just on a

different path maybe eventually they will be far more intelligent than us in everything and still will have zero Consciousness will not feel pain or

pleasure or love or hate now what adds to the problem is that there is nevertheless a very strong commercial

and political incentive to develop AIS that mimic feelings to develop AIS that can create intimate relations with human

beings that can cause human beings to be emotionally attached to the AIS even if the aiis have no feelings of themselves

they could be trained they are already trained to make us feel that they have feelings M and to start developing

relationships with them why is there such an incentive because intimacy is on the one hand maybe the most cherished thing that that a human can have that

intimacy is not a liability it's not something bad that oh I I need this no it's it's the greatest thing in the world but it's also potentially the most

powerful weapons weapon in the world if you want to convince somebody to buy a product if you want to convince somebody to vote for a certain politician or

party intimacy is like the Ultimate Weapon now is AI it is possible technically to mass produce intimacy you can create all these AIS that will

interact with us and they will understand our feelings because again feelings are also patterns You can predict a person's Feelings by watching them for weeks and months and learning

their patterns and facial expression and tone of voice and so forth and then if it's in the wrong hands it could be used to manipulate us like like never before

sure it's our ultimate vulnerability this beautiful thing that makes us human becomes this uh great weakness that we

have because as these AIS continue to self iterate their capacity to mimic Consciousness and human intimacy uh will

reach such a degree of fidelity that it will be indistinguishable to the human brain and then humans become like these unbelievably easy to hack machines who

can be directed wherever the AI you know chooses to direct them yeah it's not a a prophecy we we can take actions today to prevent this uh we can have regulations

about it we can for instance have a regulation that AIS are welcome to interact with humans but on condition that they disclose that they are AIS if

you talk with an AI doctor that's good but the AI should not pretend to be a human being you know I'm talking with an AI I mean it's not that there is no

possibility that AI will develop Consciousness we don't know I mean there could be that AI really developing it to such a degree of fidelity does it even

in terms of like how human beings interact with it does it matter for the human beings no I mean again this is the problem I mean because we don't know if they really have Consciousness or

they're only very very good at mimicking Consciousness so the key question is ultimately political and ethical if they have Consciousness if they can feel pain

and pleasure and love and hate this means that they are ethical and political subjects they have rights that uh you should not inflict pain on an AI

the same way you should not inflict pain on a human being now and the thing that it's very difficult to understand what is happening if we want humans around

the world to cooperate on this to build guard rails to regulate the development of AI first of all you need humans to understand what is happening secondly

you need the humans to trust each other and most people around the world are still not aware of what is happening on the AI front you have a very small

number of people in just a few countries mostly the US and China and a few others who understand most people in Brazil in

Nigeria in India they don't understand and this is very dangerous because it means that a few people many of them are not even elected by the US citizen they are just you know private companies they

will make the most important decisions and the even bigger problem is that even if people start to understand they don't trust each other like I the

opportunity to talk to some of the people who are leading the AI Revolution and you meet with these you know entrepreneurs and business tycoons and

politicians also in the US in China in Europe and they all tell you the same thing basically they all say we know that this thing is very very

dangerous but we can't trust the other humans if we slow down how do we know know that our competitors will also slow down whether our business competitors

let's say in here in the US or our Chinese competitors across the ocean and you go and talk with the competitors they s the same thing we know it's dangerous we would like to slow down to

give us more time to understand to assess the dangers to debate regulations but we can't we have to rush even faster because we can't trust the other

Corporation the other country and if they get it before before we get it it will be a disaster and so you have this kind of paradoxical situation where the

humans can't trust each other but they think they can trust the AIS because when you talk with the same people and you tell them okay I understand you can't trust the Chinese or you can't

trust open AI so you need to move faster developing this super AI how do you know you could trust the AI one of the things I heard you say that that really struck

me was this uh it's a quote if something ultimately destroys us it will be our own delusions H so can you elaborate on that a little bit and how that applies

to what we've been talking about yeah I mean the AI at least of the present day they cannot Escape our control and they cannot destroy us

unless we allow them or unless we kind of order them to do that we are still in control but because of our you know

political and mythological delusions we cannot trust the other humans and we think we need to develop these AIS and uh faster and faster and give them more

and more power because we have to compete with the other humans and this is the thing that could really destroy

us

[Music]