Consciousness, reasoning and the philosophy of AI | Murray Shanahan

I think there are just a huge number of enormously uh interesting philosophical questions that AI gives rise to. You

know, what is the nature of of the human mind? What is the nature of mind? What

mind? What is the nature of mind? What

about consciousness? I do think that is the wrong question and I think it's wrong in many ways. How good do you think the AI is at reasoning? Well

that's a very interesting and kind of open question and somewhat, you know controversial. You know, it really is

controversial. You know, it really is astonishing to think that every single child born today, they're going to grow up in a world where they've never known they've never known a world in which

machines can't talk to them. Welcome back to Google Deep Mind

them. Welcome back to Google Deep Mind the podcast. My guest on this episode is

the podcast. My guest on this episode is Mari Shanahan, professor of cognitive robotics at Imperial College London and principal research scientist at Google Deep Mind. Now, we have all heard the

Deep Mind. Now, we have all heard the stories about people falling in love with their chat bots, about people pushing large language models to contemplate their own existence or

questioning the limits of their conceptual understanding of reality. But

these kinds of questions about self-identity and thinking and metacognition have been puzzling philosophers for millennia already. And

so it makes sense that they should be turning to AI to interrogate the most profound questions about the nature of AI's intelligence, of its current

capabilities, even its consciousness or otherwise. Mari Shanahan has been

otherwise. Mari Shanahan has been working in the field of AI since the 1990s. And if you've been following this

1990s. And if you've been following this podcast for a while, you will remember him as the man that consulted on the 2014 science fiction film X Machina

about a computer programmer who gets the chance to test the intelligence of a female robot Ava and ultimately questions whether she is conscious.

Welcome back to the podcast, Marie. Ju

just thinking back cuz I know that you were played a key role in Ex Machina should we say, the Alex Garland film.

What do you think you got right in that film and in pre in other science fiction films that were around at the time? I

mean thinking back to sort of 10 15 years ago were we on the right track? So

one uh respect in which Xmachina really did a great service was that it does raise a whole load of very interesting and provocative questions about consciousness and about AI and

consciousness um and therefore about consciousness itself. So that's one you

consciousness itself. So that's one you know that's one huge uh success. But

it's interesting that just very shortly before Ax Macka came out, Her came out.

So Spike Jones's movie Her came out. At

the time, I really wasn't all that keen on her as a movie. Um because I just thought it was so implausible that, you know, a person could fall in love with this kind of disembodied voice, you

know, even if it's Scarlett Johansson. I

mean, how wrong was that? As a bit of prediction, I think her really did did amazingly well at predicting the world we got now. Now we don't know quite how things are going to unfold in the next

few years because because maybe robotics will uh will progress uh rapidly as well in the way that uh you know in the way that language has in AI but at the moment you know it's all about

disembodied language but also it you know her showed how people can in fact be very much um uh you know form relationships whatever but you know in the broadest sense with um disembodied

AI systems which is an extraordinary thing really. Okay, we're talking 10-15

thing really. Okay, we're talking 10-15 years ago. But but your involvement in

years ago. But but your involvement in AI goes back much further than this. You

knew John McCarthy. I did know John McCarthy. I knew him very well. John

McCarthy. I knew him very well. John

McCarthy was a professor of computer science and artificial intelligence uh back in the day. He actually coined the phrase artificial intelligence um and

was one of the authors of uh the proposal for the very famous Dartmouth conference that took place in 1956 which was the first AI conference in the world. And that conference really mapped

world. And that conference really mapped out the whole field. People just weren't thinking about this kind of thing seriously at all. It was just a handful.

So, you know, I think he was a real radical thinker and always was. Okay.

That choice of words, artificial intelligence back in 1955. Was it a good choice of words? Yeah. I mean, I still think it was. I mean, I know that some people don't, you know, think that

perhaps uh it it wasn't a good choice of words, but I still give us give us some of their arguments. So, first of all, um there is the word intelligence. So

intelligence, you know, itself is a in some ways a very contentious concept and uh you know and and especially if people think about IQ tests and and that kind

of thing and the idea that intelligence is something that can can be quantified on a on a straightforward simple scale you know, and and then some people are more intelligent than others. And I

think, you know, in in psychology, it's well recognized today that there are many different kinds of intelligence.

And this is a really important point right? there is that concern about that

right? there is that concern about that word there. So what would you have used

word there. So what would you have used differently? Well, maybe artificial

differently? Well, maybe artificial cognition or something. I often use the word cognition um to to to mean kind of you know thinking and processing information and so on. Yeah. It doesn't

have the same ring to it, does it? Let's

be honest. No. Um especially not now. I

think it's we're too far down this road aren't we? Yeah. The word artificial, I

aren't we? Yeah. The word artificial, I don't really have a problem with the word artificial. That seems like a right

word artificial. That seems like a right kind of thing. It's alluding to the fact that it's something that we've built and that hasn't evolved in nature. And so

that seems the right sort of word. The

objection to that word, I guess, is that ultimately everything that artificial intelligence is built on is at some level constructed by humans. Sure. Yes.

But it but but but it is. So what's

wrong with the word, you know, in in that case? I mean, I think that's that

that case? I mean, I think that's that that's true. Um, you were working on

that's true. Um, you were working on symbolic AI, right? Just just talk to us about the difference between that and and the other types and and where we're at now with that. Absolutely. Yeah.

Yeah. So the so-called symbolic paradigm of artificial intelligence was very much preeminent very much dominant for for decades for for many decades. So the

idea there is that um it's all about the manipulation of symbols and of you know language like sentences and uh and

symbols and using kind of reasoning processes with those symbols. So so the classic example will be an expert system. So where back in the uh you know

system. So where back in the uh you know 1980s people were building these expert systems and the idea was there was that you would try to encode medical knowledge say in a set of rules and the

rules would be something like you know oh you know if the patient uh is uh has a temperature of 104 and uh and their skin is purple and you know then there's

a 0.75% probability that they've got you know skinnyitis or something. You can

tell that I'm not a medical doctor. Yeah. And then so you'd have

doctor. Yeah. And then so you'd have thousands and thousands of these sorts of rules would be put into the into into a kind of big knowledge base and then you'd have what was called an inference

engine or you know which would carry out logical reasoning over all of these rules and there and and come to some conclusion about what the likely disease was in the in that. But it was a lot of if this then that. It was a lot of yeah

if then type type rules largely. And one

of the big problems with that is that where do the rules come from? Well

somebody has to, you know, write them all out basically. And um and so there was a whole field of knowledge elicitation where you go around to experts and you try and extract from them their understanding, you know, in

their domain, which you know, could be medical diagnosis, it could be fixing photocopers, it could be the law, and you try and codify all of this in into a

computer comprehensible very precise rule. That was a very cumbersome

rule. That was a very cumbersome process. And also what you ended up with

process. And also what you ended up with at the end was very very brittle. It

would go wrong in all kinds of ways.

Another big area of research was common sense because often it was realized that that that that we you know we implicitly have an enormous amount of common sense knowledge about the everyday world to do

with just everyday objects. They're the

the fact that they're solid, the fact that they move in certain ways. They fit

into each other in certain ways. you

know, liquids and gases and gravity and and you know, all kinds of things like that. And we actually bring all of that

that. And we actually bring all of that knowledge to bear all the time in what we're doing, but it's sort of unconscious. So then there was a big

unconscious. So then there was a big project or various big projects to try and codify all of that common sense knowledge. And then trying to turn that

knowledge. And then trying to turn that into like axioms and logic and rules and everything was a nightmare. So I

eventually I I think by about the early 2000s I I really thought that this research paradigm was kind of doomed to be honest. And I sort of you know I sort

be honest. And I sort of you know I sort of started moving away from it. But then

of course along came things like neural networks and so on. Yes. Uh which was much less about uh you know if then rules and much more about sort of extracting information from a large

amount of data. Yeah. But then I sort of wonder now about now that language is effectively cracked, have we sort of reached a higher level of abstraction

where we can we can go back to some more of those symbolic techniques, some some of those more symbolic ideas. Yeah.

Well, we certain we certainly have because nowadays one of the one of the uh you know hot topics at the moment with large language models is reasoning.

Um so you have these so-called um chain of thought models that actually carry out a whole you know they rather than simply generating an answer to a question they generate a whole uh chain

of reasoning before they issue the answer uh and that can be very very effective. So, it's interesting how that

effective. So, it's interesting how that harks back in many ways to um to the kind of thing that that people were looking at back in in the days of symbolic AI, but the underlying substrate for doing all of that is very

very different indeed because it's not hardcoded rules. It's as you mentioned

hardcoded rules. It's as you mentioned it's neural it's neural networks that have learned. Let me pick up on that

have learned. Let me pick up on that point about reasoning. As a philosopher background in logic, how good do you think the AI is at reasoning? Well

that's a very interesting and kind of open question and somewhat controversial. So, computer scientists

controversial. So, computer scientists and AI people, they have a particular notion of reasoning, a particular concept of reasoning, which very much you know, harks back to formal logic and

theorem proving. Uh, so in the days of

theorem proving. Uh, so in the days of symbolic AI for example, then you had systems that were really very good at at doing theorem proving with formal logic.

And so people think, well, that's proper reasoning. That's really that's your

reasoning. That's really that's your hardcore kind of kind of reasoning. Um

and today's uh large language models they can't match the performance of a of a, you know, a handcoded theorem prover, you know, or logic engine of the sort that's been

around for decades. Give me an example of a type of theorem that that might be able to be proved by a hard-coded system. So, it will be where you've got

system. So, it will be where you've got maybe uh maybe, you know, 20 or 30 axioms of logic. And so, it might be something like the number that follows one is two. Well, uh I mean it could be something like that. It could be in the

in the domain of number theory or something very mathematical. But it

could be something much more much more everyday. So for example, suppose that

everyday. So for example, suppose that you've got some very difficult logistical planning problem where maybe you have hundreds of lorries and and depots and goods and and and all kinds

of things like that. And you need to plan the routes and the deployment of the lorries and where they're going to go. So that that that's a very kind of

go. So that that that's a very kind of difficult problem computationally and it can be expressed very precisely in in in in formal rules and that's the kind of situation where you might want to use a

a good old-fashioned straightforward algorithm uh planning algorithm of the sort that's been around for a long time now. Contemporary large language models

now. Contemporary large language models are getting better and better at this kind of thing, but they're still, you know, you don't have those kinds of mathematical guarantees that they're always going to come up with exactly the right answer. And it's very easy to to

right answer. And it's very easy to to to kind of make examples where you have more and more axioms and so on um where they're going to going to slip up.

There's a whole separate research direction which is to try and build more handcoded things that combine today's AI techniques with more old-fashioned uh symbolic techniques to specifically for

mathematical theorem proving. And Deep

Mind has done some amazing work along those lines. But that's different from

those lines. But that's different from large language models. with large

language models, we're thinking of a of these chat bots that um that that can talk about anything under the sun. And

one of the things they happen to be able to do is a kind of reasoning. So that

kind of that's that's not going to be at the moment quite as good as you could do by hand building something for that.

It's kind of interesting because because hand building something is I mean you end up with something that's very rigid.

That's the problem. Yeah. And and

brittle. Yes. Absolutely. But then at the same time, the sort of flexibility that you get from from the genative AI approach, you you know, it's it's too floppy as it were. You know, you want the rigidity in there. Well, you know

maybe or maybe not. I mean, I think many examples of human affairs are just not as as uh black and white as that. And

you know, you do maybe want things to be a bit more blurry even in sort of simple everyday things like, you know, what would be good flowers to put over in this corner of the garden? Well, you

know, we've already got got some roses in that corner there, and those roses are yellow, so we'd have but we can't have too much yellow, so we maybe we'd need to move them to the other corner of the garden. But then at the same time

the garden. But then at the same time though, is this is this real reasoning or is this just the AI kind of mimicking well ststructured arguments

that have existed in the training data but just in a in a sort of novel novel environment? Yeah. Well, of course, that

environment? Yeah. Well, of course, that begs the question, what is real reasoning? I don't think there's it's

reasoning? I don't think there's it's it's not written in the sky, you know what real reasoning is. It's up to us to define the concept of of real of real reasoning or of reasoning. And so so we

have that, you know, we were talking earlier on about kind of mathematical reasoning of the sort that logicians do and that was you is done by kind of theorem proves in the past and so on and

and today. Um but that's you know that's

and today. Um but that's you know that's when people were first using the terms like reasoning they weren't thinking of that kind of thing. And when we use the word reasoning in everyday life, we're not thinking about that sort of thing.

So if you're chatting away to a large language model and uh about your garden and you sort of say, I'm thinking about what plants are and and it says, well you know, maybe you should consider this kind of plant in that kind of location because that's best for the soil and

given you said that the winds, you know it's windy there and you know, we would just say that that is supplying reasons.

I mean, it is supplying reasons for now where they come from is another matter.

So people might say, well, it's just mimicking what's in the in the training set, but you know, it's probably never seen exactly that example exactly before. So it's it's moving beyond the

before. So it's it's moving beyond the training set to a certain extent. And I

think it's just using the everyday concept of reasoning in an everyday way to call that reasoning. I'm just

thinking back to some of the the different characteristics that the earlier philosophers wanted artificial intelligence to have and reasoning being being one of them. But then but then also the cheuring test which of course

you know gets brought up all the time about a way to test for the capability of of an artificial intelligence. I mean

it's kind of controversial, right? I

suppose in terms of how good it ever would have been as a test for the capability of AI. How what was your take on it? Do you think it was ever a good

on it? Do you think it was ever a good test? No, I I've always thought it was a

test? No, I I've always thought it was a terrible test, but a but a but a really great um spur to philosophical discussion about about things. And

again, with a bit of hindsight, maybe I might backtrack on a little bit on a few few of my views because I I was certainly very very much of the opinion that embodiment was a critical uh facet

of intelligence was critical for achieving you know intelligence which doesn't come anywhere near the Turing test at all. Right. No, the cheuring test is is absolutely explicitly nothing to do with embodiment because because

the in the cheuring test so just to remind people what it what it is. So you

have so in the cheuring test you have kind of two subjects as it were. One is

a human and the other is the computer and then you have a judge. The human

judge can't see you know which is uh which is the computer and which is the humans and they're only talking to these subjects um through a kind of chat-like interface. They can't see that they're

interface. They can't see that they're they're whether they're embodied or not.

So there um we can you know easily suppose that the uh that the computer might be one of today's large language models in which case you know I have to say that today they would pretty much would pass the juring test you know I mean we've got to that point which is

amazing really but so so I used to think that it was a bad test because it didn't test any of these embodied skills so so you'd need a robot really to test whether something was capable of the

kind of everyday cognition that we all uh put to use when we're for example making a cup of tea or something because otherwise it's a very very narrow narrow form of intelligence. Yes, it's all to

do with language and and reasoning and not to do with the kinds of things that evolution was, you know, developed in us and in other animals before before language, right? Which is the ability to

language, right? Which is the ability to manipulate and move around with and and navigate and and uh exploit, you know in the best sense of the word, the everyday physical world. So, actually

that's really interesting. That's so

interesting because I I often think about how fine maybe the the large language models we have at the moment can pass the cheing test but they don't flinch if you throw a ball at your computer. Oh no indeed. And in a sense

computer. Oh no indeed. And in a sense there are these sort of as you say these much deeper forms. Maybe we wouldn't class them as intelligence in a in in in the way that we talk about it but but

ultimately they it sort of is a form of intelligence. I think it very much is a

intelligence. I think it very much is a form of intelligence. Moreover, I I think that in the biological case, and now I have to caveat all these things by saying in the biological case, you know our ability to to to to think and to

reason and to talk is very much grounded in our interaction with the everyday world. If you think about

world. If you think about almost all of your everyday uh speech is using spatial metaphors. I mean, they completely permeate uh our everyday speech. Even the word permeate. Perate.

speech. Even the word permeate. Perate.

Yeah, absolutely. All grounded. I use

the grounded, you know. So, so, so we just use those kinds of things all all the time because we're fundamentally physical beings. Because we're

physical beings. Because we're fundamentally physical beings and because our brains have evolved to help us to to survive and reproduce and in in in in you know in this physical world.

Yeah. Uh and while interacting with all these other beings that are doing the same thing, right? Because there are some alternatives when you are trying to test for the capability of an artificial intelligence.

Just talk me through some of the potential alternatives that we have.

Well, I think perhaps you've got in mind the Garland test, what I call the Garland test, which is um uh so that goes back to the film X Machina, which was directed by Alex Garland, of course.

And there's a bit in the script where Nathan, the billionaire guy, uh is talking to Caleb, and Caleb, uh who's the, you know, the guy who's been brought in to to to interact with Ava

the robot, and Caleb says, "Oh, I'm here to kind of conduct a Turing test on Ava." And Nathan says, "Oh, no. We're

Ava." And Nathan says, "Oh, no. We're

way past that. Ava could pass the Turing test easily." The point is to show you

test easily." The point is to show you she's a robot and see if you still think she's conscious. Wow. And that's what I

she's conscious. Wow. And that's what I call the Garland test. And it's

different from the Turing test in two respects. So, first of all, the sort of

respects. So, first of all, the sort of judge, as it were, which who in that case is um uh is Caleb, can see that she's a robot. So, in the che in the cheuring test, the judge can't see which

is which. But here the idea is is that

is which. But here the idea is is that is that you know Caleb sees knows that she's knows that she's her brain is an AI brain and yet still attributes these

characteristics to her. And the

characteristic in question also is different because it's not intelligence it's not can she think but is she conscious or is it conscious is she which is an entirely different test and I think you know intelligence and consciousness are different things and

we can disentangle those two things dissociate them. Um, so so when I first

dissociate them. Um, so so when I first read the script of the film and that those particular lines were in there for Caleb and Nathan and and I wrote next to it in my version spot on an exclamation

mark because I just thought Alex totally nailed a really important idea there and so in my writing I call this the Garland test and quite a few people have picked up on that and called it the Garland

test as well. Is there a test that would really impress you if an AI were able of passing it? So I always was very

passing it? So I always was very impressed with Francois Shere's uh ARC um uh tests and that's ARC which stands for abstract reasoning corpus. So these

are little um sequences of images uh of the sort that you know you get in IQ tests and things um and they're the images are arranged in pairs. So so you have the first image it's kind of

pixelated image. It's got little, you

pixelated image. It's got little, you know, cells and with little kind of um things that you can interpret as objects or lines and so on in in the images and you're interested in the challenge is to

work out a rule that takes you from one image to the second one. Then you've got to apply that rule to a third image.

First of all, he held out and made completely secret all of the test ones.

So you couldn't game it by kind of knowing what the the the actual test versions were or using it in a training set. Or using it in the training set.

set. Or using it in the training set.

That's that's sort of what I mean yeah by gaming and also the he very carefully designed them so that it was very different rules each each rule you know was completely different to to the other

rules and you usually had to find some kind of intuitive application of often our everyday common sense knowledge is seeing this as like a liquid that's moving in this direction or imagining

this thing moving you know growing or something. So it required grounding in a

something. So it required grounding in a way. Well, it seemed to, but uh but

way. Well, it seemed to, but uh but recently, you know, people have been able to make significant progress on on on these in a in a more um brute force

kind of way. Um so uh so so I don't I I feel that the solutions are um are not really, you know, um getting at the spirit of the original test quite so much. Well, that's it. I guess in a way

much. Well, that's it. I guess in a way is that as soon as you as soon as you set a metric, as soon as you set a bar for once we've crossed this this threshold, then we will have capability

intelligence, consciousness, whatever it might be, it it it sort of changes the the the whole nature of the test in itself. People are going to start, you

itself. People are going to start, you know, the test, right? It's it's

goodart's law, right? So, absolutely. A

lot of people who come on this podcast have have sort of expressed real need for caution about anthropomorphizing these things. Are you one of those

these things. Are you one of those people who who thinks that we shouldn't?

Well, I think there are different ways of looking at this and I think there are there are there are sort of good and bad forms of anthropomorphization. So, so on on the one hand, people can uh start to

form relationships as they see it with uh with with AI systems, friendships and companionships and mentorships and uh you know that that

can potentially be a bad thing if they are misled into thinking that they can trust the thing that they're talking to or that they're really in love with it or that it really cares about them. On

the other end of the spectrum, if u an AI system is just using the word I, then I think that that's a pretty harmless um sort of form of selfanthropomorphization. We even see

selfanthropomorphization. We even see buses that say things like on the side like I am out of service and we don't have a problem with that kind of thing.

So I don't see why we should have a problem with that with large language models either. But you know, I think we

models either. But you know, I think we do tend to anthropomorphize things. When

we had satnavs in cars that weren't just in our phones, I used to anthropomorphize the satnav all the time. I used to think, oh, you know

time. I used to think, oh, you know stupid thing. it thinks we're doing this

stupid thing. it thinks we're doing this and it's a natural human tendency I think. What about the other words that

think. What about the other words that we use? I mean the example that you gave

we use? I mean the example that you gave of the satn saying oh it thinks we're in the car park or oh it believes that this is it got this wrong. It misunderstood

this. Those are all very human centric words, aren't they? Yeah. Yeah.

Absolutely. So that so that what um they're examples of what philosophers often call folk psychology. So we have this folk psychology where we use words like belief, concepts like belief

desire and intention which we can apply to not just to um other humans and and other animals but we can apply to you know objects as as well. It's what the

philosopher Dan Dennett uh called the in taking the intentional stance. So we

adopt the intentional stance towards something. If we talk about it and think

something. If we talk about it and think about it as if it acted on the basis of having beliefs and goals and carrying out rational decisions for what it does

on the on the basis of those things and uh and that's a very useful way of thinking about many many things such as even our satnav or a chess computer. So

so for Dan Denny, that was one of the examples that he used a chess computer that oh, you know, it wants to get the the queen forward because it thinks I'm going to use my rook to to defend this

rank or something. And that's full of of this kind of intentional folk psychological language about beliefs and goals and things. Is that problematic?

Then if we start using that that that idea of beliefs and intentions and desires about the AI. So it's only problematic if we start to use these

things in ways that mislead us into thinking that that things have capa capabilities that they don't really have. So I think that's where it becomes

have. So I think that's where it becomes uh problematic. say um the encyclopedia

uh problematic. say um the encyclopedia bratannica right the the the physical volume of the encyclopedia bratannica doesn't know that Argentina won the world cup in this because it's too it's

too it's too old so if if you made that remark it would make perfect sense you know you you might say that and it's fine if somebody said to you why don't you have a conversation with it about

England's football uh prowess you know or lack thereof you know that would be ridiculous right now the interesting thing is that now we've got these large language models you can have a conversation with them about you can say

tell it things and so so that it kind of pushes the boundary of where of where we we might start to say well it doesn't really X Y Z it pushes that a little bit further out I wonder if there's

something even deeper here about this this human need or or maybe it's just a desire to really um want AI to to have

these characteristics to be anthropomorphized yeah well that's a really interesting uh question isn't it so I don't think it kind of comes comes back to that. It does. It comes back to language. You know, in this case, we're

language. You know, in this case, we're inclined to anthropomorphize things because they're really good at using language. And for us, the only things

language. And for us, the only things that are good at using language are other humans. And so, it's very strange

other humans. And so, it's very strange in a way to be suddenly in a world where we have language using things that, you know, it's not just humans that can talk. That's astonishing. Yeah. I mean

talk. That's astonishing. Yeah. I mean

it is astonishing. It is astonishing.

And what's you know it's really is astonishing to think that every single child born today they're going to grow up in a world where they've never known they've never known a world in which

machines can't talk to them. Isn't that

an extraordinary thing? Um yeah I mean it really is. And uh so what the implications are of that for us all is really hard to say. I'm just thinking

back to to what you were saying about how grounded humans are in the physical world. Yes. It does feel like

physical world. Yes. It does feel like the kind of embodied aspect of AI has lagged behind this language aspect quite

a bit. Yeah. Do you think that we're

a bit. Yeah. Do you think that we're going to see a big upstep in intelligence, however you want to define it, or or broader capabilities once we

get good and effective embodied AI?

Well, I think it might make a big difference because the large language models we have at the moment, it's really difficult to discern actually to be honest right now where the limits are

for how good they're going to get whether we really are on the road to producing, you know, general intelligence that's comparable to human general general intelligence. And uh and

often you know when you you you get to the sort of the boundaries of the capabilities of these kinds of things you sort of get sometimes you get the impression that the AI system doesn't really quite grock you know something it

doesn't really deeply understand something you reach some kind of limit and you realize that it's that it it's been faking it a little bit but it may be that that sort of general ability to

really kind of get things on a deep level on a deep you know kind of common sense level maybe but that does still require a bit of uh embodiment. It just

does still basically require training data that involves you know interacting with a real world of physical objects with their spatial organization and there's something fundamental about

that. Okay. If understanding then

that. Okay. If understanding then however we define it is something that can emerge as a just a consequence of of more and more data. What about

consciousness? I mean, I'm sure you've been asked a thousand times about AI consciousness and and whether it's something that we can expect to happen or or has already happened. Yeah. Yeah.

The very first thing to point out is that I do think we can dissociate, you know, intelligence or cognition and cognitive capabilities. We can

cognitive capabilities. We can dissociate that from consciousness. So I

think we can imagine things that are uh very capable and have you know that we want to say are very intelligent uh because of the way they can achieve their goals and so on but that we don't

want to ascribe consciousness to but actually what does that even mean to to ascribe consciousness to to something at all and I think the concept of consciousness itself uh you know can be

broken down into many parts it's a it's a multifaceted concept so for example we might talk about awareness of the world and many in in the scientific study of consciousness there are all of these

experimental protocols and paradigms and many of them are to do with perception you know and and you're looking at whether a person is aware of something is consciously perceiving something in

in the world large language models are not aware of the world at all in that in that respect but there are other facets of consciousness we also have self-awareness and our self-awareness part of that is awareness of our own

body and where it is in in in in space but another aspect to self-awareness is a kind of awareness of our own you know you know machinations of our stream of consciousness as William James called

it. So we have a we have that kind of

it. So we have a we have that kind of self-awareness as well and we have what some people call metacognition as well.

We have the ability to think about what we know and then additionally there's the emotional side or the feeling side of consciousness or sentience. So the

capacity to feel the capacity to to to suffer. Um, and that's another aspect of

suffer. Um, and that's another aspect of consciousness. Now, I think we can

consciousness. Now, I think we can dissociate all of these things. Now, in

humans, they all come as a big package a big bundle. We only actually have to think about non-human animals to realize that we can kind of start to separate these things uh these things a little bit because I think that much as I love

uh cats, I think there's a limited self-awareness going on in cats. Well, you know, I'm a big cat

cats. Well, you know, I'm a big cat person, I have to say. So, I I do say that with some hesitation. And you know there's little metacognition, shall we say? Yeah. Certainly they don't have an

say? Yeah. Certainly they don't have an awareness of their own ongoing stream of verbal consciousness because they don't they don't have it. So they're so they're not they're not thinking about what they did yesterday in verbal terms

or what they want to do with their lives. So if we think about like robots

lives. So if we think about like robots you may have a very sophisticated robot you know, even your robot vacuum cleaner and you may say that it's well, you know, it does actually have a kind of awareness of the world. And that's not

an inappropriate use of that phrase awareness of the world. Do I want to call it consciousness? Well, then I seem to be bringing on board all of this other stuff as well, but you don't have to. You can break down the concept of

to. You can break down the concept of consciousness into these different aspects because your robot vacuum can know exactly where it is in a space and how. Yeah. And respond in a, you know

how. Yeah. And respond in a, you know in an intelligent and sensitive way to where it is and the objects around it and achieve its ends and so on. So

there's a kind of awareness of the world there. I don't there's no

there. I don't there's no self-awareness. There's certainly no

self-awareness. There's certainly no capacity for suffering. And so in a large language model, there might not be awareness of the world in that perceptual sense, but maybe there's some

kind of like in sort of selfawareness or reflexive capabilities, reflexive cognitive capabilities. They can talk

cognitive capabilities. They can talk about the things that they've talked about earlier in the conversation, for example, and and can do so in a you know, in a reflective manner, which kind

of feels a little bit like some aspects of self-awareness that that we have a little bit. I don't think that it's

little bit. I don't think that it's appropriate to think of them in terms of of having feelings. They can't

experience pain because they don't have a body. I think we can take the concept

a body. I think we can take the concept apart basically. So then is the question

apart basically. So then is the question can AI be conscious or not as though it's a binary thing. It's it's the wrong question from the off. I do think that is the wrong question and I think it's

wrong in many ways. So, so just then we were talking about the fact that it's actually a sort of multifaceted concept but also I think that we tend to have these these very deep metaphysical uh

commitments to the idea of consciousness as some you know sort of magical thing that uh that is you know a metaphysical thing. So the question of whether

thing. So the question of whether something is conscious or not is not a matter of of you know consensus or a matter of just our language but it's something that is that is out there in the metaphysical reality or in the mind

of God or in the platonic heaven or something like that. But ultimately I do think that that's the wrong way of thinking about consciousness. Let's take

one aspect of consciousness then that that you described about the sort of emotional side an ability to suffer but not necessarily physical pain emotional pain too and sort of a sense of self in

the emotional way. Do you think this is something that will just emerge as a natural consequence of intelligence that if you build something that is intelligent enough at some point this is going to

happen or is there something unique about biological creatures and I guess the process of evolution that we've been through that has resulted in that that can't be replicated in a machine. I

don't think there is a right or wrong answer to your question there. You know

I think we just have to wait and see what things uh we bring into the world and and how we end up treating them and talking about them and thinking about them. And I don't think we really know

them. And I don't think we really know until until they're among us, as it were, you know, these things that we're building. Then then we will just be led

building. Then then we will just be led to think about them and talk about them and treat them in a in in in a in a particular way. So an example I like to

particular way. So an example I like to think of in this regard is the octopus.

Mhm. So, so octopuses have recently been brought into, you know, UK legislation brought into the category of things that we have to care about the welfare of.

That's as a result of of lots of things I think, happening. So, the public has been exposed to being with octopuses a lot more now. And not you don't have to literally be under the water and and and

and poking around with octopuses to to to know what it's like to be with them because there's all kinds of wonderful documentaries and wonderful books by like Peter Godfrey Smith has these great books about interacting with octopuses

and so on. And and so so so those sort of narratives and documentaries. They

give us a feel for what it's like to be with an octopus, what it's like to have an encounter with an with an octopus.

And then you know you can't you sort of can't help yourself but to see it as a fellow conscious creature you know but complementing that is the scientific progress as well. So at the same time

scientists study the the nervous systems of uh of octopuses and you know realize the extent to which their nervous systems are similar to ours and the way

that that when that we experience pain you can find analogous you know aspects of their nervous systems to to ours. So

taking all these things together, I think that tends to affect the way we think about them and the way we talk about them and way the way we treat them. So I think the same kind of thing

them. So I think the same kind of thing will, you know, is going to happen with AI systems. Do I think there's a right or wrong answer to to, you know, could could we be misled there? I think that's

a really really deep and difficult metaphysical philosophical question. I

do wonder though I mean that that point about suffering to me seems different to the others because because metacognition you know the sort of sense of the world etc. There's not these ethical

implications necessarily about those but but I think with suffering like you wouldn't want your shoes to be conscious you know you wouldn't want a forklift truck to be sort of conscious unless they happen to really like being a

forklift truck. Sure. But then do we

forklift truck. Sure. But then do we have to be a tiny bit more careful about that particular aspect of it?

If there were the prospect of being something that is genuinely capable of suffering then we should think very hard about whether we should do it or not. You know I tend to think that that's not the case with anything

that we've got at the moment. But you

know some people uh will will push back against that if we take the example of large language models. Well okay so there's one level in which um what they do is next token prediction, next word

prediction. But in order to be able to

prediction. But in order to be able to do that, you know, really really really well in the way that that they can at the moment, then all they've had to

learn, you know, and and acquire all kinds of emergent mechanisms. So who knows whether or not there's some kind of emergent mechanism has been learned in the weights of this enormous staggeringly huge number, hundreds of

billions of weights in a in a language model. whether some mechanism is hasn't

model. whether some mechanism is hasn't been learned there that you know has for example genuine understanding in it whatever that means or even

consciousness coming back to embodiment again I've always uh been of the view that it's only really legitimate to talk about consciousness in the context of something we can share a world with and

and and and and have that kind of encounter with that we have with an octopus or a dog or a horse or whatever and being together in the world with that that animal and responding to

things together, then I'm in no doubt that they are conscious. That's a kind of primal case for me. Now, with a large language model, you can't be in the same world as them in that kind of way, and

you can't hang out with them and interact with physical objects. With

today's large language models, right?

So, to my mind, using the language of consciousness in that context is what Vicinstein would would say it's taking language on holiday. It's using it out so far outside of its normal use. you

know, maybe it's inappropriate, but that can change, you know, and I and and the more I interact with with large language models, the more I have these sophisticated and interesting conversations with them, the more I'm

inclined to think, well, maybe I want to extend the language of consciousness bend it, change it, distort it, make up some new words, break it apart in ways that are going to fit the these new

things that I'm interacting with all the time. I know you've spent a lot of time

time. I know you've spent a lot of time interacting with these large language models. I've I've actually seen you

models. I've I've actually seen you described as a renowned prompt whisperer. What's your secret? Well, one

whisperer. What's your secret? Well, one

secret is to talk to the large language model as if it were human. So, if you think that what they're doing is roleplaying a a human character, such as say say uh you know a very smart and

helpful intern, then you should treat them like a smart and helpful intern and talk to them as if they were a smart and helpful intern. For example, just being

helpful intern. For example, just being polite and saying uh you know is that clear and please and thank you. And in

my experience, you get better responses out of things if you if you do do things that way. Do you say please and thank

that way. Do you say please and thank you? You can say please and thank you.

you? You can say please and thank you.

Yeah. Now, there's a good reason, good scientific reason why that might get you know, again, it just depends. And

you know, models are changing all the time why that might get better uh per better performance out of it. Because if

it's roleplaying, say it's roleplaying a super a very smart intern, right? uh

then then they might then it's going to just roleplay maybe being a bit more stroppy if they don't if they're not being treated politely. It's you know it's just mimicking what humans would do

you know in those in that that scenario.

So the mimicry might extend to kind of being a bit more you know not not being as responsive uh if the if their boss is uh is a bit is a bit of a stroppy you

know so bossy boss. I absolutely love that. I think I want to return to where

that. I think I want to return to where we started which is about how we think about AI and the language we use to describe it and sort of how we kind of

frame it in our minds. Do you think that we need a new way of talking about AI? I

do both acknowledges its potential without overestimating it but then similarly isn't dismissive of of of the things that it can do. I think that's exactly what we need. In one of my

papers, I use the phrase exotic mindlike entities to to to describe large language models. So I think that they

language models. So I think that they are to a degree exotic mindlike entities. So they are so they are kind

entities. So they are so they are kind of mindlike and they're increasingly mindlike. Now there's a very important

mindlike. Now there's a very important reason for using the little hyphen like there, which is because I I I want to hedge my bets as to whether they really qualify as minds. And so I can wrigle

out of that problem by just using mind like. They're exotic because they're not

like. They're exotic because they're not like us language use, but in other respects they're disembodied for a start. There's really weird conceptions

start. There's really weird conceptions of selfhood that are applicable to them maybe, but so they are quite exotic entities as well. So they're ex I think of them as exotic mindlike entities and

we just don't have um the right kind of conceptual framework and vocabulary for talking about these exotic mindlike entities yet. you know, we're we're

entities yet. you know, we're we're working on it and um uh and and the more they are around us, the more we'll develop new kinds of ways of talking and and and thinking about them. It is

interesting though that you are still going for the sort of the cheering like approach of like a creature almost rather than the the the idea. Well, you

know, an entity is a pretty neutral term, isn't it? I suppose you could just say thing, exotic mindlike thing if you prefer. Yeah, let's go with that. I

prefer. Yeah, let's go with that. I

think let's uh let's let's push for that for the for the the new N. Okay. Okay.

But I mean I you know I can't Hannah because I've used the word entity in that context like in many publications now. So exotic mindlike entities. I like

now. So exotic mindlike entities. I like

it. I like it a lot. Thank you so much for joining us. It's been a pleasure Hannah. Thank you. One of the nice

Hannah. Thank you. One of the nice things about having done this podcast for a number of years is that you really get to see how the people at the frontier of AI, how their opinions

change and shift over time. And the last few years have been a real gamecher in all sorts of ways. about the extent to

which intelligence requires a physical body. About how much we need to expand

body. About how much we need to expand our definition of consciousness to account for the subtly different ways that these mindlike entities can operate

in the next few years. Well, who knows?

But if past predictions are any indication, the only thing we know about tomorrow's science and technology is that it will be radically different to what we imagine today.

You have been listening to Google DeepMind the podcast with me, Professor Hannah Fry. If you enjoyed this episode

Hannah Fry. If you enjoyed this episode then do subscribe to our YouTube channel. You can also find us on your

channel. You can also find us on your favorite podcast platform. And of

course, we have plenty more episodes on a whole range of topics to come. So, do

check those out. See you next time.