Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation | Lex Fridman Podcast #376

you know I can tell chat gbt create a piece of code and then just run it on my computer and I'm like you know that that sort of personalizes for me the what could what could possibly go wrong so to

speak was that exciting or scary that possibility it was a little bit scary actually because it's kind of like if you do that right what is the sandboxing that you should have and that's sort of

a that's a a version of of that question for the world that is as soon as you put the AIS in charge of things you know how much how many constraints should there be on these systems before you put the

AIS in charge of all the weapons and all these you know all these different kinds of systems well here's the fun part about sandboxes is uh the AI knows about

them it has the tools to uh crack them the following is a conversation with Stephen Wolfram his fourth time on this podcast he's a computer scientist mathematician theoretical physicist and

the founder of Wolfram research a company behind Mathematica well from alpha or from language and the Wolfram physics and meta mathematics projects He

has been a Pioneer in exploring the computational nature of reality and so he's the perfect person to explore with together the new quickly evolving

landscape of large language models as human civilization Journeys towards building super intelligent AGI this is the Lex Friedman podcast to support it please check out our sponsors

in the description and now dear friends here's Stephen Wolfram you've announced the integration of chat gbt and Wu from Alpha and Wolfram

language so let's talk about that integration what are the key differences from the high philosophical level maybe the technical level between the capabilities of

broadly speaking the two kinds of systems large language models and this computational gigantic computational system infrastructure that is well from alpha yeah so what does something like

chat GPT do it's it's mostly focused on make language like the language that humans have made and put on the web and so on yeah so you know it's it's primary

sort of underlying technical thing is you've given a prompt it's trying to continue that prompt in a way that somehow typical of what it's seen based

on a trillion words of text that humans have written on the web and the way it's doing that is with something which is probably quite similar to the way we

humans do the first stages of that using a neural net and so on and just saying given these given this piece of text let's Ripple through the neural net one

word and get one word at a time of output and uh it's kind of a shallow computation on a large amount of kind of training data that is what we humans

have put on the web that's a different thing from sort of the computational stack that I spent the last I don't know 40 years or so building which has to do

with what can you compute many steps potentially a very deep computation it's not sort of taking the statistics of what we humans have produced and trying

to continue Things based on that statistics instead it's trying to take kind of the formal structure that we've created in our civilization whether it's from mathematics or whether it's from

kind of systematic knowledge of all kinds and use that to do arbitrarily deep computations to figure out things that that aren't just let's match what's

already been kind of said on the web but let's potentially be able to compute something new and different that's never been computed before so as a practical matter you know the the um what we're

you know the our goal is to have made as much as possible of the world computable in the sense that if there's a question that in principle is answerable from some sort of expert knowledge that's

been accumulated we can compute the answer to that question and we can do it in a sort of reliable way that's that's the best one can do given what the expertise that our civilization has

accumulated it's a very it's a it's a much more sort of labor-intensive on the side of kind of being creating kind of the the computational system to do that

um obviously the in the the kind of the chat GPT world it's like take things which were produced for quite other purposes namely the all the things we've

written out on the web and so on and sort of forage from that things which were are like what's been written on the web so I think you know as a practical point of view I I view sort of the chat

GPT thing as being wide and shallow and what we're trying to do with sort of building out computation as being this sort of deep also broad but but most

importantly kind of a deep type of thing I think another way to think about this is if you go back in human history you know I don't know a thousand years or something and you say what what can the

typical person what's the typical person going to figure out well the answer is there's certain kinds of things that we humans can quickly figure out that's sort of what what our uh you know other

neural architecture and the kinds of things we learn in our lives let us do but then there's this whole layer of kind of formalization that got developed in which is you know the kind of whole

sort of story of intellectual history and the whole kind of depth of learning that formalization turned into things like logic mathematics science and so on

and that's the kind of thing that allows one to kind of build these towers of of uh uh of of uh sort of towers of things you work out it's not just I can

immediately figure this out it's no I can use this kind of form to go step by step and work out something which was not immediately obvious to me and that's kind of the story of what what we're

trying to do computationally is to be able to build those kind of tall towers of what implies what implies what and so on um and uh as opposed to kind of the yes

I can immediately figure it out it's just like what I saw somewhere else in something that I heard or remembered or something like this what can you say about the kind of formal structure the

kind of form of foundation you can build such a formal structure on about the kinds of things you would start on in order to build this kind of

uh deep computable knowledge trees so the question is sort of how do you how do you think about computation and there's there's a couple of points here

one is what computation intrinsically is like and the other is what aspects of computation we humans with our minds and with the kinds of things we've learned

can sort of relate to in that computational universe so if we start on the kind of what can computation be like it's something I've spent some big chunk

of my life studying is imagine that you're you know we usually we write programs where we kind of know what we want the program to do and we carefully write you know many lines of code and we hope that the program does what we what

we intended it to do but the thing I've been interested in is if you just look at the kind of natural science of programs so you just say I'm going to make this program it's a really tiny program maybe I even pick the pieces of

the program at random but it's really tiny by really tiny I mean you know less than a line of code type thing you say what does this program do and you run it and big discovery that I made in the

early 80s is that even extremely simple programs when you run them can do really complicated things really surprised me it took me several years to kind of realize that that was a thing so to

speak but that that realization that even very simple programs can do incredibly complicated things that we very much don't expect that Discovery I mean I realized that that's very much I think how nature

works that is nature has simple rules but yet does all sorts of complicated things that we might not expect you know as a big thing of the last few years has been understanding that that's how the

whole universe and physics works but that's a a quite separate topic but so there's this whole world of programs and what they do and very rich sophisticated things that these programs can do but

when we look at many of these programs we look at them and say well that's kind of I don't really know what that's doing it's not a very human kind of thing so on the one hand we have sort of what's possible in the computational

universe on the other hand we have the kinds of things that we humans think about the kinds of things that are developed in kind of our intellectual history and that's uh and the Really the

challenge to sort of making things computational is to connect what's computationally possible out in the computational universe with the things that we humans sort of typically think

about with our minds now that's a complicated kind of moving Target because the things that we think about change over time we've learned more stuff we've invented mathematics we've

invented various kinds of ideas and structures and so on so it's gradually expanding we're kind of gradually colonizing more and more of this kind of intellectual space of possibilities but

the the real thing the real challenge is how do you take what is computationally possible how do you take how do you encapsulate the kinds of things that we think about in a way that kind of plugs

into what's computationally possible and and actually the the uh the big sort of idea there is this idea of kind of symbolic programming symbolic representations of things and so the the

question is when you look at sort of everything in the world and you kind of you know you take some visual scene or something you're looking at and you say well how do I turn that into something that I can kind of stuff into my mind

you know there are lots of pixels in my visual scene but the things that I remembered from that visual scene are you know there's a there's a chair in this place it's a kind of a symbolic

representation of the visual scene there are two chairs on a table or something rather than there are all these pixels arranged in all these detailed ways and so the question then is how do you take sort of all all the things in the world

and make some kind of representation that corresponds to the types of ways that we think about things and human language is is sort of one form of representation that we have we talk

about chairs that's a word in human language and so on how do we how do we take but human language is is not in and of itself something from that plugs in

very well to sort of computation it's not something from which you can immediately compute consequences and so on and so you have to kind of find a way

to take sort of the the stuff we understand from human language and make it more precise and that's really this story of of symbolic programming and you

know what what that turns into is something which I didn't know at the time it was going to work as well as it has but back in the 1979 or so I was trying to build my first big computer system and trying to figure out you know

how should I represent computations at a high level and I kind of invented this idea of using kind of symbolic Expressions you know structured as it's

kind of like a a function and a bunch of arguments but that function doesn't necessarily evaluate to anything it's just a a thing that sits there

representing a structure and so building up that structure and it's turned out that structure has been extremely it's it's a good match for the way that we

humans it seems to be a good match for the way that we humans kind of conceptualize higher level things and it's been the last I don't know 45 years or something it's served me remarkably

well so building up that structure using this kind of symbolic representation but what can you say about abstractions here because you could just start with your physics project you could start at a

hypograph at a very very low level and build up everything from there but you don't you type shortcuts right uh you take the highest level of abstraction convert that

uh the kind of abstraction that's convertible to something computable using symbolic representation and then that's your new foundation for that little piece of knowledge yeah

somehow all of that is integrated right so the the sort of a very important phenomenon that that is kind of a thing that I've sort of realized is just it's one of these things that sort of in the

in the future of kind of everything is going to become more and more important is this phenomenon of computational irreducibility and the the question is if you know the rules for something you

have a program you're going to run it you might say I know the rules great I know everything about what's going to happen well in principle you do because you can just run those rules out and just see

what they do you might run them a million steps you see what happens Etc the question is can you like immediately jump ahead and say I know it's going to happen after a million steps and the

answer is 13 or something yes and the the one of the very critical things to realize is if you could reduce that computation there isn't a sense no point in doing the computation the place where

you really get value out of doing computation is when you had to do the computation to find out the answer but this phenomenon that you have to do the computation to find out the answer this phenomenon of computational

irreducibility seems to be tremendously important for thinking about lots of kinds of things so one of the things that happens is okay you've got a model of the universe at the low level in terms of atoms of space and hypographs

and rewriting typographs and so on and it's happening you know 10 to the 100 times every second let's say well you say great then we've we've nailed it we've we know how the universe works

well the problem is the universe can figure out what it's going to do it does those 10 to the 100 you know steps but for us to work out what it's going to do we have no way to reduce that

computation the only way to do the computation to see the result of the computation is to do it and if we're operating within the universe we're kind of there's no there's no opportunity to do that because the universe is doing it

as fast as the universe can do it and that's you know that's what's happening so what we're trying to do and a lot of the story of science a lot of other kinds of things is finding pockets of

reducibility that is you could have a situation where everything in the world is full of computational irreducibility we never know what's going to happen next the only way we can figure out what's going to happen next is just let

the system run and see what happens so in a sense the story of of most kinds of science inventions a lot of kinds of things is the story of finding these places where we can locally jump ahead

and one of the features of computational reducibility is that there are always pockets of reducibility there are always places there always an infinite number of places where you can jump ahead there's no way where you can jump

completely ahead but there are little little patches little places where you can jump ahead a bit and I think you know we can talk about physics project and so on but I think the thing we

realize is we kind of exist in a slice of all the possible computational irreducibility in the universe we exist in a slice where there's a reasonable amount of predictability and in a sense

as we try and construct these kind of higher levels of of abstraction symbolic representations and so on what we're doing is we're finding these lumps of

reducibility that we can kind of attach ourselves to and about which we can kind of have fairly simple narrative things to say because in principle you know I say what's going to happen in the next

few seconds you know oh there are these molecules moving around in the air in this room and oh gosh it's an incredibly complicated story um and that's a whole computational irresistible thing most of which I don't

care about and most of it is well you know the air is still going to be here and nothing much is going to be different about it and that's a kind of reducible fact about what is ultimately

a an underlying level of computational irreducible process and uh life would not be possible if we didn't have a large number of such

reducible Pockets uh yes Pockets amenable to uh reduction into something symbolic yes I think so I mean life in

in the way that we experience it that I mean you know one might you know depending on what we mean by life so to speak the the the experience that we

have of sort of consistent things happening in the world the idea of space for example where there's you know we can just say you're here you move there it's kind of the same thing it's still

you in that different place even though you're made of different atoms of space and so on this is this idea that it's uh that there's sort of this level of predictability of what's going on that's

us finding a slice of reducibility in what is underneath this computationally reducible kind of system and I think that's that's sort of the thing which is

actually my favorite discovery of the last few years is the realization that it is sort of the interaction between this sort of underlying computational

irreducibility and our nature as kind of observers who sort of have to key into computational reducibility that fact leads to the main laws of physics that

we discovered in throughout his century so this is we talked about this in in more detail but this is a uh to me it's kind of our nature as observers the fact

that we are computationally bounded observers we don't get to follow all those little pieces of computational irreducibility to stuff what is about out there in the world into our minds

requires that we are looking at things that are reducible we are compressing kind of we're extracting just some Essence some kind of symbolic essence of what's the detail of what's going on in

the world that together with one other condition that at first seems sort of trivial but isn't which is that we believe we are persistent in time

that is yes you know uh so some sense of causality here's the thing at every moment according to our Theory we're made of different atoms of space

at every moment sort of the microscopic detail of what what the universe is made of is being Rewritten and that's and in fact the very fact that there's coherence between different parts of space is a consequence of the fact that

there are all these little processes going on that kind of knit together the structure of space it's kind of like if you wanted to have a fluid with a bunch of molecules in it if those molecules weren't interacting you wouldn't have this fluid that would pour and do all

these kinds of things it would just be sort of a free-floating collection of molecules so similarities with space that the fact that space is kind of knitted together as a consequence of all

this activity in space and the fact that kind of what we consist of sort of this this series of of uh you know we're continually being Rewritten and the

question is why is it the case that we think of ourselves as being the same us through time that's kind of a key assumption I think it's a key aspect of what we see as sort of our Consciousness

so to speak is that we have this kind of consistent thread of experience well isn't that just another limitation

of our mind that we want to reduce reality into some that kind of temporal yeah consistency is just a nice narrative right tell ourselves well the

fact is I think it's critical to the way we humans typically operate is that we have a single thread of experience you know if you if you imagine sort of a mind where you have you know maybe

that's what's happening in various kinds of Minds that aren't working the same way other minds work is that you're splitting into multiple threads of experience it's also it's also something where you know when you look at I don't

know Quantum Mechanics for example in the insides of quantum mechanics it's splitting into many threads of experience but in order for us humans to interact with it you kind of have to

have to knit all those different threads together so that we say oh yeah a definite thing happened and now the next definite thing happens and so on and I think you know sort of inside uh it's

it's sort of interesting to try and imagine what's it like to have kind of these uh fundamentally multiple threads of experience going on I mean right now different human Minds have different

threads of experience we just have a bunch of Minds that are interacting with each other but we don't have a you know within each mind there's a single thread and that's a that is indeed a simplification I think it's a it's a

thing you know the general computational system does not have that simplification and um it's one of the things you know I I people often seem to think that you know Consciousness is the highest level

of kind of things that can happen in the universe so to speak but I think that's not true I think it's actually a a specialization in which among other things you have this idea of a single

threat of experience which is not a general feature of anything that could kind of computationally happen in the universe so it's a feature of a computationally limited system that's only able to

observe reducible Pockets so yeah so I mean this word Observer it means something in quantum mechanics it means something

in a lot of places it means something to us humans right as conscious beings so what what's the importance of the Observer what is the Observer and what's the importance of the observer in the

computational universe so this question of what is an observer what's the general idea of an observer it's actually one of my next projects which got somewhat derailed by the the current sort of AI Mania but um is there a

connection there or is that uh do you do you think the Observer is primarily a physics phenomena is it related to the whole AI thing yes yes it is related so one question is what is a general

Observer so you know we know we have an idea what is a general computational system we think about Turing machines we think about other models of computation there's a question what is a general

model of an observer and the there's kind of observers like us which is kind of The Observers we're interested in you know we could imagine an alien Observer that deals with computational irreducibility and it has a mind that's

utterly different from ours and and completely incoherent with what what we're like but the fact is that that you know if we are talking about observers like us that one of the key things is

this idea of kind of taking all the detail of the world and being able to stuff it into a mind being able to take all the detail and kind of you know

extract out of it a smaller set of of kind of degrees of freedom a smaller number of elements that will sort of fit in our minds and I think this this question so I've been interested in

trying to characterize what is the general Observer and the general Observer is I think in part there are many let me give an example of a you know you have a gas it's got a bunch of

molecules bouncing around and the thing you're measuring about the gas is its pressure and the only anything you as an observer care about is pressure and that means you have a piston on the side of

this box and the Piston is being pushed by the gas and there are many many different ways that molecules can hit that piston but all that matters is the kind of aggregate of all those molecular

impacts because that's what determines pressure so there's a huge number of different configurations of the gas which are all equivalent so I think one key aspect of observers is this equivalency of many different

configurations of a system saying all I care about is this aggregate feature all I care about is this this overall thing and that's that sort of one one aspect and when we see that in lots of

different again it's the same story over and over again that there's there's a lot of detail in the world but what we are extracting from it is something a sort of a thin a thin summary of that of

that detail is that thin summary nevertheless true is can it be a crappy approximation sure that on average is is correct I mean if we look at the

Observer that's the human mind it seems like there's a lot of very um as represented by natural language for example there's a lot of really crappy approximation sure and that could

be maybe a feature of it well with this ambiguity right right you don't know you know it could be the case you're just measuring the aggregate impacts of these molecules but there is some tiny tiny

probability that molecules will arrange themselves in some really funky way and that just measuring that average isn't going to be the main point yeah by the way an awful lot of science is very

confused about this because you know you look at you look at papers and people are really Keen they draw this curve and they have these you know these bars on the curve and things it's just this

curve and it's this one thing and it's supposed to represent some system that has all kinds of details in it and this is a way that lots of science has gotten wrong because people say I remember

years ago I was studying snowflake growth you know you have the Snowflake and it's growing it has all these arms it's doing complicated things but there was a literature on this stuff and it talked about you know what's the rate of

snowflake growth and you know it got pretty good answers for the rate of the growth of the Snowflake and they looked at it more carefully and they had these nice curves of you know snowflake growth rates and so on I looked at it more carefully and I realized according to

their models the snowflake will be spherical and so they got the growth rate right but the detail was just utterly wrong and you know that not only the detail

that the whole thing was was not capturing you know it was capturing this aspect of the system that was in a sense missing the main point of what was going

on and what is the geometric uh shape of a snowflake snowflakes start in in the phase of water that's relevant to the formation of snowflakes it's a phase of

ice which starts with a hexagonal arrangement of of water molecules and so it starts off growing as a hexagonal plate and then what happens is is the plate oh oh versus sphere sphere well no

no but it's it's much more than that I mean snowflakes are fluffy you know typical snowflakes have little little dendritic Arts yeah and what actually happens is it's kind of kind of cool because you can make these very simple

discrete models with cellular automata and things that that figure this out you start off with this you know hexagonal thing and then the places it starts to

grow little arms and every time a little piece of ice it adds itself to the snowflake the fact that that ice condensed from the water vapor heats the

snowflake up locally and so it makes it less likely for uh for another piece of ice to accumulate right nearby so this leads to a kind of growth inhibition so

you grow an arm and it is a separated arm because right around the arm it got a little bit hot and it didn't add more ice there so what happens is it grows

you have a hexagon it grows out arms the arms go arms and then the arms go arms go arms and eventually actually it's kind of cool because it actually fills in another hexagon a bigger hexagon and when I first looked at this we had a

very simple model for this I realized you know when it fills in that hexagon it actually leaves some holes behind so I thought well you know that is that really right so look at these pictures of snowflakes and sure enough they have these little holes in them that are kind

of scars of the way that these arms grow out um so you can't fill in backfill holes yeah they don't backfill and presumably

there's a limitation of how big like you can't arbitrarily grow I'm not sure I mean the thing falls through the the I mean I think it this you know it hits the ground at some point I think you can grow I think you

can grow in the lab I think you can grow pretty big ones I think you can grow many many iterations of this kind of goes from hexagon it grows out arms it turns back it fills back into a hexagon

it grows more arms again in 3D no it's flat usually why is it flat why doesn't it uh span out okay wait a minute you said it's fluffy and fluffy is a

three-dimensional property no or no it's it's fluffy snow is okay so you know what makes we're really uh we're really in it

it's multiple snowflakes become fluffy a single snowflake is not fluffy no no single snowflake is Fluffy and what happens is you know if if you have snow

that it's just pure hexagons they they can you know they they fit together pretty well it's not it doesn't it doesn't make it doesn't have a lot of air in it and they can also slide against each other pretty easily and so

the snow can be pretty you know can I think avalanches happen sometimes when when the things tend to be these you know hexagonal plates and it kind of slides but then when the thing has all

these arms that have grown out it's not they don't fit together very well and that's why the snow has lots of air in it and if you look at one of these snowflakes and if you catch one you'll see it has these little arms and people

actually people often say you know no two snowflakes are alike um that's mostly because as a snowflake grows they do grow pretty consistently with these different arms and so on but

you capture them at different times as they you know they fell through through the air in a different way you'll catch this one at this stage and as it goes through different stages they look really different and so that's why you

know kind of looks like no two slime flakes are alike because you caught them at different at different times so the rules under which they grow are the same it's just the timing is yes okay so the

point is science is not able to uh describe the full complexity of snowflake growth well science if you if you do what people might often do just

say okay let's make it scientific let's turn into one number and that one number is kind of the growth rate of the arms or some such other thing that fails to capture sort of the detail of what's

going on inside the system and that's in a sense a big challenge for science is how do you extract from the natural world for example those aspects of it

that you are interested in talking about now you might just say I don't really care about the fluffiness of the snowflakes all I care about is the growth rate of the arms in which case you know you have you can have a good

model without knowing anything about the fluffiness um but the fact is as a practical you know when if you if you say what's the what is the most obvious feature of a snowflake oh that it has this

complicated shape well then you've got a different story about what you model I mean this is one of the features of sort of modeling and science that you know what is a Model A model is some way of

reducing the actuality of the world to something where you can readily sort of give a narrative for what's happening where you can basically make some kind of abstraction of what's happening and

answer questions that you care about answering if you want to answer all possible questions about the system you'd have to have the whole system because you might care about this particular molecule where did it go and you know your model which is some big

abstraction of that has nothing to say about that so you know one of the things that's that's often confusing in science is people will have I've got a model somebody says somebody else will say I don't believe in your model because it

doesn't capture the feature of the system that I care about you know there's always this controversy about you know is the is it a correct model well no model is a except for the actual system itself is a correct model in the

sense that it captures everything questions does It capture what you care about capturing sometimes that's ultimately defined by what you're going to build technology out of things like this the one counter example to this is

if you think you're modeling the whole universe all the way down then there is a notion of a correct model but even that is more complicated because it depends on kind of how observers sample

things and so on that's a that's a separate story but at least at the first level to say you know this thing about oh it's an approximation you're capturing one aspect you're not capturing other aspects when you really

think you have a complete model for the whole universe you better be capturing ultimately everything even though oh to actually run that model is impossible because of computational reducibility

the only the only thing that successfully runs that model is the actual running of the universe is the universe itself but okay so what you care about is an interesting concept so that's a

that's a human concept so that's what you're doing with uh wolf from Alpha and Wolfram language is you trying to come up with symbolic representations yes as

simple as possible uh so a model that's as simple as possible that fully captures stuff we care about yes so I mean for example you

know we could we'll have a thing about you know data about movies let's say we could be describing every individual pixel in every movie and so on but that's not the level that people care

about and it's yes this is a I mean and and that level that people care about is somewhat related to what's described in natural language but what what we're

trying to do is to find a way to sort of represent precisely so you can compute things see see one thing when you say you give a piece of natural language question is you feed it to a computer you say does the computer understand

this natural language well you know the computer process it in some way it does this maybe it can make a continuation of the natural language you know maybe it can go on from The Prompt and say what it's going to say

you say does it really understand it hard to know but for in this kind of computational world there is a very definite definition of does it

understand which is could it be turned into this symbolic computational thing from which you can compute all kinds of consequences and that's the that's the sense in which one has sort of a target

for the understanding of natural language and that's kind of our goal is to have as much as possible about the world that can be computed in a in a

reasonable way so to speak be able to be sort of captured by this kind of computational language that's that's kind of the goal and and I think for us humans the the main thing that's

important is as we formalize what we're talking about it gives us a way of kind of building a structure where we can sort of build this Tower of consequences of things so if we're just saying well

let's talk about it in natural language it doesn't really give us some hard Foundation that lets us you know build step by step to work something out I mean it's kind of like what happens in math if we were just sort of vaguely

talking about math but didn't have the kind of full structure of math and all that kind of thing we wouldn't be able to build this kind of big tower of consequences and so you know in a sense what we're trying to do with the whole

computational language effort is to make a formalism for describing the world that makes it possible to kind of build this this Tower of consequences well can

you talk about this dance between natural language and Wolfram language so there's this gigantic thing called

the internet where people post memes and diary type thoughts and very important sounding articles and all of that that makes up the training data set for GPT

and then there's a wolf from language how can you map from the natural language of the internet to the Wolfram language is there an

manual is there an automated way of doing that as we look into the future well so wolf from alpha what it does it's kind of front end is turning

natural language into computational language right what you mean by that is there's a prompt you ask a question what is the capital of some yeah and it turns into you know what's the distance

between you know Chicago and London or something and that will turn into you know geo-distance of entity City you know Etc et cetera Etc each one of those things is very is very well defined we

know you know given that it's the entity City Chicago et cetera et cetera et cetera you know Illinois United States you know we know the geolocation of that we know it's population we know all

kinds of things about it which we have you know curated that data to be able to to know that with some degree of certainty so to speak and then

then we can compute things from this and that's that's kind of the um yeah that's that's that's the idea but then something like GPT large

language models do they allow you to uh make that conversion much more powerful okay so that's an interesting thing which we still don't know everything about okay the um I mean this question

of going from natural language to computational language yes in will from alpha we've now you know wolfenovo's been out and about for what 13 and a half years now and you know we've

achieved I don't know what it is 98 99 success on queries that get put into it now obviously there's a sort of feedback loop because the things that work are things people go on putting into it so

that that um uh but you know we've got to a very high success rate of the the little fragments of natural language that put people put in you know questions math calculations chemistry

calculations whatever it is you know we can we can we we do very well at that turning those things into to computational language now I from the very beginning of Orphan Alpha I thought

about for example uh writing code with natural language in fact I had a I was just looking at this recently I had a post that I wrote in 2010 2011 called something like programming with natural

language is actually going to work okay and so you know we had done a bunch of experiments using methods that were a little bit some of them a little bit

machine learning like but certainly not the same you know the same kind of idea of vast training data and so on that is the story of large language models actually I know that that post a piece

of utter trivia but that that post um uh Steve Jobs forwarded that post around to all kinds of people at Apple you know that was because he never really liked programming languages so he was very

happy to see the idea that that that that you could get rid of this kind of layer of kind of engineering like structure he would have liked you know I think what's happening now because it

really is the case that you can you know this idea that you have to kind of learn how the computer works to use a programming language is something that is I think a a thing that you know just

like you had to learn the details of the op codes to know how Assembly Language worked and so on it's kind of a thing that's that's that's a limited time Horizon but but kind of the the you know

so this idea of how elaborate can you make kind of the prompt how elaborate can you make the natural language and Abstract from

it computational language it's a very interesting question and you know what chat gbt you know gbt4 and so on can do is pretty good

um it isn't it's very interesting process I'm still trying to understand this workflow we've been working out a lot of tooling around this workflow the natural language to computational language right and the process

especially if it's conversation like dialogue it's like multiple queries kind of thing yeah right there's so many things that are really interesting that that work and so on so first thing is

can you just walk up to the computer and expect to sort of specify a computation what one realizes is humans have to have some idea of kind of this way of thinking about things computationally

without that you're kind of out of luck because you just have no idea what you're going to walk up to a computer I remember when I should tell a silly story about myself the very first computer I saw which is when I was 10

years old it was a big Mainframe computer and so on and I didn't really understand what computers did and it's like somebody's showing me this computer and it's like uh you know can the computer work out the weight of a

dinosaur it's like that isn't a sensible thing to ask that's kind of you know you have to give it that's not what computers do I mean in Wolfram Alpha for example you could say what's the typical weight of a Stegosaurus and we'll give

you some answer but that's a very different kind of thing from what one thinks of computers as doing and so the the kind of the the question of uh you know first thing is people have to have

an idea of what what computation is about um you know I think it's a very you know for Education that is the key thing it's kind of this this sir this notion not

computer science not so the details of programming but just this idea of how do you think about the world computationally computation thinking about the world computationally is kind

of this formal way of thinking about the world we've had other ones like logic was a formal way you know as a way of sort of abstracting and formalizing some aspects of the world mathematics is another one computation is this very

broad way of sort of formalizing the way we think about the world and the thing that's that's cool about computation is if we can successfully formalize things in terms of computation computers can

help us figure out what the consequences are it's not like you formalized it with math well that's nice but now you have to if you're you know not using a computer to do the math you have to go

work out a bunch of stuff yourself so I think but that this idea let's see I mean that you know we're trying to take kind of the we're talking about sort of natural language and its relationship to

computational language the uh the thing the sort of the typical workflow I think is first human has to have some kind of idea of what they're trying to do that if if it's something that they want to

sort of build a tower of of capabilities on something that they want to sort of formalize and make computational so then human can type something in to you know

some llm system and uh uh sort of say vaguely what they want in sort of computational terms then it does pretty well at synthesizing wealth language

code and it'll probably do better in the future because we've got a huge number of examples of of natural language input together with the wolf and language translation of that so it's kind of a a

um uh you know that that's a thing where you can kind of extrapolating from all those examples uh makes it easier to do that that toss is the prompter task could also kind of debug in the from

language code or is your hope to not do that debugging no no no I mean so so there are many steps here okay so first the first thing is you type natural language it

generates woven language give examples by the way you have an example that is the the dinosaur example do you have an example that jumps to mind that we should be thinking about some dumb

example it's like take my heart rate data and uh you know figure out whether I uh you know make a moving average

every seven days or something and work out what the um and make a plot of the result okay so that's a thing which is you know about two-thirds of a line of language

code I mean it's you know list plot of moving average of some data bin or something of the of the data and then you'll get the result um and you know the vague thing that I

was just saying in natural language could would almost certainly correctly turn into that very simple piece of language code so you start mumbling about heart rate

yeah and it kind of you know you arrive at the moving average kind of idea but you say average over seven days maybe it'll figure out that that's a moving you know that that can be encapsulated

as this moving average idea I'm not sure but then the typical workflow but I'm seeing is you generate this piece of often language code it's pretty small usually

um it's uh and if it isn't small it probably isn't right but um you know if it's it's pretty small and you know welcome language is one of the ideas of open languages it's a language that

humans can read it's not a language which you know programming languages tend to be this one-way story of humans write them and computers execute from them orphan language is intended to be

something which is sort of like math notation something where you know humans write it and humans are supposed to read it as well and so kind of the workflow

that's emerging is kind of this this human mumbles some things you know large language model produces a fragment of awesome language code then you look at

that you say yeah that looks well typically you just run it first you see does it produce the right thing you look at what it produces you might say that's obviously crazy you look at the code you

see I see why it's crazy you fix it if you really care about the result you really want to make sure it's right you better look at that code and understand it because that's the way you have this sort of checkpoint of did it really do

what I expected it to do now you go beyond that I mean it's it's it's you know what we find is for example let's say the code does the wrong thing then you can often say to the large language

model can you adjust this to do this and it's pretty good at doing that interesting so you're using the output of the code

to give you hints about the the function of the code so you're debugging yeah based on the output of the code itself right the

plug-in that we have the the you know for chat GPT it does that routinely you know it will send the thing in it will get a result it will discover the llm will discover itself that the result is

not plausible and it will go back and say oh I'm sorry it's very polite and it you know it goes back and says I'll rewrite that piece of code and then it will try it again and get the result the other thing is pretty interesting is

when you're just running so one of the new Concepts that we have we invented this whole idea of notebooks back 36 years ago now and so now there's the question of sort of how do you combine

this idea of notebooks where you have you know text and code and output how do you combine that with the notion of of chat and so on and there's some really interesting things there like for

example a very typical thing now is we have these these notebooks where as soon as the if if the thing produce uses errors if the you know run this code and

it produces messages and so on the the llm automatically not only looks at those messages it can also see all kinds of internal information about stack traces and things like this and it can

then it does a remarkably good job of guessing what's wrong and telling you so in other words it's it's looking at things sort of interesting it's kind of a typical sort of ai-ish thing that it's able to have

more sensory data than we humans are able to have because they're able to look at a bunch of stuff that we humans would kind of glaze over looking at and it's able to then come up with oh this is the explanation of what's happening

and and what is the data the stack trace the the code you've written previously the natural language you've written yeah it's also what's happening is one of the things that's uh is is for example when there's these messages there's documentation about these messages

there's examples of where the messages have occurred otherwise nice all these kinds of things the other thing that's really amusing with this is when it makes a mistake one of the things that's in our prompt when the code doesn't work

is read the documentation and we have a you know another piece of the plugin that lets it read documentation and that again is very very useful because it it will you know

it will figure out sometimes it'll get it'll make up the name of some option for some function that doesn't really exist read the documentation it'll have you know some wrong structure for the

function and so on it's um that's a powerful thing I mean the thing that you know I've realized is we built this language over the course of all these years to be nice and coherent and consistent and so on so it's easy for

humans to understand turns out there was a side effect that I didn't anticipate which is it makes it easy for AIS to understand so it's almost like another natural language but yeah so so what

formal language is a kind of foreign language yes yes you have a lineup English French Japanese Wolfram language and then uh I don't know Spanish and

then the system is not going to notice well yes I mean maybe you know that's an interesting question because it really depends on what I see as being a a

important piece of fundamental science that basically just jumped out at us with Chachi BT um because I think you know the the real question is why does chat GPD work how

is it possible to encapsulate you know to successfully reproduce all these kinds of things in natural language um you know with a you know a comparatively small he says you know a

couple hundred billion you know weights of neural net and so on and I think that uh you know that that relates to kind of a fundamental fact about language which uh you know the the main the main thing

is that I think there's a structure to language that we haven't kind of really explored very well as kind of the semantic grammar I'm talking about about

um about language I mean we kind of know that when we when we set up human language we know that it has certain regularities we know that it has a certain grammatical structure you know

noun followed by verb followed by noun adjectives Etc et cetera et cetera that's its kind of grammatical structure but I think the thing that chat gbt is

showing us is that as an additional kind of regularity to language which has to do with the meaning of the language Beyond just this pure you know part of speech combination type of thing and I

think the uh the the kind of the the one example of that that we've had in the past is logic and you know I I think my my sort of uh uh uh kind of picture of

how was logic invented how was logic discovered uh it really was the thing that was discovered in its original conception it was discovered presumably by Aristotle who kind of listened to a

bunch of people orators you know giving speeches and this one made sense that one doesn't make sense this one and you know you see these patterns of you know

if the uh you know I don't know what you know if the uh if the Persians do this then this does that Etc et cetera et cetera and what what Aristotle realized is there's a

structure to those sentences there's a structure to that rhetoric that doesn't matter whether it's the Persians and the Greeks or whether it's the cats and the dogs it's just you know p and Q you can

abstract from this the the details of these particular sentences you can lift out this kind of formal structure and that's what logic is that's a heck of a discovery by the way logic you're making

me realize now yeah it's not obvious the fact that there is an abstraction from natural language that has where you can fill in

any word you want yeah is a very interesting Discovery now it took a long time to mature I mean Aristotle had this idea of syllogistic logic where there were these particular patterns of how

you could argue things so to speak and you know in the Middle Ages part of Education was you memorize the syllogisms I forget how many there were but 15 of them or something and they all had names they all had mnemonics like I

think Barbara and celerant were two of the the mnemonics for the the syllogisms and people would kind of this is a valid argument because it follows the Barbara syllogism so to speak and and it took

until 1830 um you know with uh George boole to kind of get beyond that and kind of see that there was a a level of abstraction that was beyond the this particular template

of a sentence so to speak um and that's you know what what's interesting there is in a sense you know you know Chachi BT is operating at the Aristotelian level it's essentially

dealing with templates of sentences by the time you get to Bool and Boolean algebra and this idea of you know you can have arbitrary depth nested collections of ands and ores and Knots

and you can resolve what they mean um that's the kind of thing that's a computation story that's you know you've gone beyond the pure sort of templates of natural language to something which

is an arbitrarily deep computation but the thing that I think we realize from from chat GPT is you know Aristotle stopped too quickly and there was more that you could have lifted out of

language as formal structures and I think there's you know in a sense we've captured some of that in in you know some of what what is in language that there's there's a there's a lot of kind

of little calculator little algebras of of what you can say what language talks about I mean whether it's I don't know if you say uh

I go from place a to place B Place B to place C then I know I've gone from place a to place C if a is a friend of B and B is a friend of C it doesn't necessarily

follow that a is a friend of C these are things that are uh you know that there are if if you go from from place a to place B plus b to place C it doesn't matter how you went like logic it

doesn't matter whether you flew there walked there swam there whatever you still this transitivity of of where you go is still valid and there are there

are many kinds of kind of features I think of the way the world Works uh that are captured in these aspects of of language so to speak and I think what

what chat gbt effectively has found just like it discovered logic you know people are really surprised it can do these these logical inferences it discovered Logic the same way Aristotle discovered logic by looking at a lot of sentences

effectively and noticing the patterns in those sentences but it feels like it's discovering something much more complicated than logic so this kind of semantic grammar I think he wrote about this

um maybe we can call it the laws of language I believe you call or which I like the laws of thought yes that was the title that George boole had for his

for his Boolean algebra back in 1830 but yes I was a thought yes that was what he said all right so he thought he thought he nailed it with blue in algebra yeah

there's more to it yeah it's a good question how much more is there to it and it seems like one of the reasons as you uh imply that the

reason gbt Works chat GPT works is that uh there's a finite number of things to it yeah I mean it's discovering the laws in

some sense GPT is discovering this laws of semantic grammar that underlies language yes what's sort of interesting is in the computational universe there's

a lot of other kinds of computation that you could do they're just not ones that we humans have cared about and and operate with and that's probably because our brains are built in a certain way

and you know the neural Nets of our brains are not that different in some sense from the neural Nets of of uh of a large language model and that's kind of and and so when we think about and you know maybe we can talk about this some

more but when we think about sort of what will AIS ultimately do the answer is insofar as AIS are just doing computation they can run off and do all these kinds of crazy computations

but the ones that we sort of have have decided we care about are there is this kind of very limited set that's where the uh reinforcement learning with human

feedback seems to come in the more the AI say the stuff that kind of interests us the more we're impressed by it so you can do a lot of interesting intelligent things but we're only

interested in the AI systems when they communicate human in a human-like way you ask about human-like topics yes well it's it's like technology I mean in a

sense the physical world provides all kinds of things you know there's all kinds of processes going on in physics only a limited set of those are ones that we capture and use for technology

because they're only a limited set but we say you know this is a thing that we can sort of apply to the human purposes we currently care about I mean you might have said okay you pick up a piece of of

rock you say okay this is a nice silicate it contains all kinds of silicon I don't care then you realize oh we could actually turn this into a you know semiconductor wafer and make it

microprocessor out of it and then we care a lot about it yes um and it's it's you know it's this thing about what do we you know in the evolution of our civilization what things do we identify

as being things we care about I mean it's like you know when when there was a little announcement recently of the possibility of a high temperature superconductor that involved you know the element lieutium which you know

generally nobody has cared about and you know it's kind of um but suddenly if there's this application that relates to kind of human purposes we start to care

a lot so given your thinking that GPT may have discovered inklings of laws of thought do you think such laws exist can we Linger on that what's your intuition

here oh definitely I mean the fact is look the the logic is but the first step there are many other kinds of calculi

about things that uh we consider you know about sort of things that happen in the world or things that are meaningful well how do you know logic is not the last step you know what I mean so

because we can plainly see that that thing I mean if you say here's a sentence that is syntactically correct okay you look at it and you're like you

know the happy electron you know eight I don't know what some something that it just it you look at and it's like this is meaningless it's just a bunch of words it's syntactically correct the

nouns and the verbs are in the right place but it just doesn't mean anything um and so there clearly is some rule that there are rules that determine when

a sentence is has the potential to be meaningful that go beyond the pure parts of speech syntax and so the question is what are those rules and are there fairly finite set of those rules my

guess is that there's a fairly finite set of those rules and they you know once you have those rules you have a kind of a construction kit just like this the rules of syntactic grammar give

you a construction kit for making syntactically correct sentences so you can also have a construction kit for making semantically correct sentences those sentences may not be realized in

the world I mean I think you know the elephant flew to the moon yeah a a syntactic a semantically you know we know we have an idea if I say that to

you you kind of know what that means but the fact is it hasn't been realized in the world so to speak so semantically correct perhaps there's things that can be imagined with the human mind no uh

things that are consistent with both our imagination and our understanding of physical reality I don't yeah good question I mean it's a

good question it's a good question I mean I think it is it is given the way we have constructed language it is things which which fit with the things

we're describing in language it's a bit circular in the end because you know you can and and the and the the sort of boundaries of what is physically

realizable okay let's take the example of motion okay motion is a complicated concept it might seem like it's a concept that should have been figured out by the Greeks you know long ago but

it's actually really pretty complicated concept because what is motion motion is you can go from place a to place B and it's still you when you get to the other

end right you you take an object you move it and it's still the same object but it's in a different place now even in ordinary physics that doesn't always work that way if you're near a

space-time singularity in a black hole for example and you take your teapot or something you don't have much of a teapot by the time it's near the space-time Singularity it's been completely you know deformed beyond

recognition but so that's a case where pure emotion doesn't really work you can't have a thing stay the same but so this idea of motion is is something that

sort of is a slightly complicated idea but once you have the idea of motion you can start once you have the idea that you're going to describe things as being

the same thing but in a different place that sort of abstracted idea then has you know that has all sorts of consequences like this transitivity of motion go from A to B B to C you've gone

from a to c um and that's so that level of description you can have what are sort of uh inevitable consequences they're inevitable features of the way you've

sort of set things up and that's I think what this sort of semantic grammar is capturing is things things like that and I you know I think that it's a question of what does the word mean when you say

I go from I move from here to there well it's complicated to say what that means this is this whole issue of you know is pure motion possible et cetera et cetera et cetera but once you have kind of got

an idea of what that means then there are inevitable consequences of that idea but the very idea of meaning it seems like there's some words that become

um it's like there's a latent ambiguity to them I mean it's the word like emotionally loaded words like hate and love

right it's like what what are they what do they mean exactly like what um so especially when you have relationships between complicated objects we seem to take this kind of

shortcut descriptive shortcut of to describe like right object a hates object B what's that really mean right well words are defined by kind of our

social use of them I mean it's not you know a word in computational language for example when we say we have a a construct there

we expect that that construct is a building block from which we can construct an arbitrarily tall tower so we have to have a very solid building block and you know we have to it turns into a piece of code it has

documentation it's you know it's a whole it's a whole thing but the word hate you know the documentation for that word well there isn't a standard documentation for that word so to speak

it's a complicated thing defined by kind of how we use it when you know if it wasn't for the fact that we were using language I mean so so what is language at some level language is a way of

packaging thoughts so that we can communicate them to another mind can these complicated words be converted into something that a

computation engine can use right so so I think the answer to that is that that what one can do in computational language is Define make a def make a

specific definition and if you have a complicated word like let's say the word eat okay you'd think that's a simple word it's you know animals eat things whatever else but you know you do

programming you say this function eats arguments which is sort of poetically similar to the animal eating things but if you start to say well what are the

implications of you know uh the function eating something you know does it can can the function be poisoned well maybe it can actually but um uh you know if there's a tight mismatch or something in

some language but but you know in what how far does that analogy go and it's it's just an analogy whereas if you use the word eat in a computational language

level you would Define there isn't a thing which you anchor to the kind of natural language concept eat but it is now some precise definition of that that then you can compute things from but

don't you think the analogy is also per se software eats the world don't you think there's a there's something Concrete in terms of

meaning about analogies sure but the thing that sort of is the first Target for computational language is to take sort of the ordinary meaning of things

and try and make it precise make it sufficiently precise you can build these towers of computation on top of it so it's kind of like if you start with a piece of poetry and you say I'm going to

Define my program with this piece of poetry it's kind of like that's that's a difficult thing it's better to say I'm going to just have this boring piece of prose and it's using words in the

ordinary way and that time communicating with my computer and that's how I'm going to build the solid building block from which I can construct this whole kind of computational Tower so there's

some sense where if you take a poem and reduce it to something computable you're going to have very few things left so maybe there's a bunch of human interaction that's just poetic

aimless nonsense well that's just like recreational like hamster in a wheel it's not actually producing anything well I I I think that

that's a complicated thing because in a sense human linguistic communication is there's one mind it's producing language that language is having an effect on

another mind yeah and the question of there's sort of a a type of effect that is well defined let's say where where for example it's very independent of the two minds that the it doesn't you know

that there there's communication where it can matter a lot sort of what the experience of of um uh of one mind is versus another one and so on

yeah but uh what is the purpose of natural English communication well I think I think versus so computation computational language

somehow feels more amenable to the definition of purpose it's like yeah you're given to clean representations of a concept and

you can build a tower based on that is is natural language the same thing but more fuzzy what well I think the the story of natural language right in the the that's the great invention of our

species we don't know whether exists in other species but we know it exists in our species it's the thing that allows you to sort of communicate abstractly

from like one generation of the species to another you can you know there is an abstract version of knowledge that can be passed down it doesn't have to be you know genetics it doesn't have to be you know you don't have to Apprentice the

next species you know the next generation of birds to the previous one to show them how something works yeah there is this abstracted version of knowledge that can be kind of passed

down now that you know it relies on it still tends to rely because language is fuzzy it does tend to rely on the fact that you know if we look at the you know some ancient language that where we

don't have a chain of translations from it until what we have today we may not understand that ancient language um and we may not understand you know its Concepts may be different from the ones that we have today we still have to

have something of a chain but it is something where we can realistically expect to communicate abstract ideas and that's you know that's one of the big big roles of a language I think you know

in in um uh it's you know that that's been this this ability to sort of concretify

abstract things is what what language has provided do you see natural language and thought as the same the stuff that's going inside your mind well

that's been a long debate in philosophy it seems to be become more important now when we think about how intelligent GPT is whatever that means whatever that means

but it seems like the stuff that's going on in the human mind seems something like intelligence is language but we call it intelligence yeah we call it well yes and so you start to think of okay what's the relationship between

thought the language of thought the laws of thought the laws of the words like reasoning and the laws of language and how that

has to do with computation which seems like a more rigorous precise ways of reasoning right which are Beyond human I mean much of what computers do human

humans do not do I mean you might say humans are a subset yeah presumably yes hopefully yes the the yes right you know you might say who needs computation when

we have large language models large language models can just you know eventually you'll have a big enough neuron that it can do anything but they're really doing the kinds of things that humans quickly do and there are

plenty of sort of formal things that humans never quickly do for example I don't know I you know you can some people can do mental arithmetic they can do a certain amount of math in their in their minds I don't think many people

can run a program in their minds of any sophistication it's just not something people do it's not something people have even thought of doing because just it's kind of a it's kind of not you know you

can easily run it on a computer when another portray program yeah aren't we running specialized programs yeah yeah but if I say to you here's a turing machine yeah you know tell me what it

does after 50 steps and you're like trying to think about that in your mind that's really hard to do it's not what people do I mean well in some sense people program they build a computer

they program it just to answer your question about what the system does after 50 steps I mean humans build computers yes yes yeah that's right but

they've created something which is then you know then when they run it it's doing something different than what's happening in their minds I mean they've outsourced that that piece of

computation from something that is internally happening in their minds to something that is now a tool that's external to their minds so whether we're humans to you didn't invent computers

they discovered them they discovered computation which they invented the technology of computers this the computer is just a

kind of way to plug into this whole this stream of computation which probably other are the ways for sure I mean the the you know the particular ways that we make computers out of semiconductors and

electronics and so on that's the particular technology stack we built I mean the story of a lot of what people try to do with Quantum Computing is finding different sort of underlying

physical you know infrastructure for doing computation you know biology does lots of computation it does it using an infrastructure that's different from semiconductors and electronics it's a

you know it's a molecular scale uh sort of computational process that hopefully we'll understand more about I have some ideas about understanding more about that but uh you know that's that's another ins you know it's another

representation of computation things that happen in the physical Universe at the level of you know these evolving hypographs and so on that's another sort of implementation layer for this

abstract idea of computation so if GPT or a large language models starting to form starting to develop or implicitly understand the laws of language and

thought do you think they can be made explicit yes how okay he had a bunch of effort I mean so do they are doing Natural Science I mean what is happening

in Natural Science you have the world that's doing all these complicated things and then you discover you know Newton's laws for example this is how motion works this is the way that this particular sort of idealization of the

world this is how we describe it in a simple computationally reducible way and I think it's the same thing here it's there are sort of computationally reducible aspects of what's Happening

that you can get a kind of narrative theory for just as we've got narrative theories in physics and so on God

do you think it will be depressing or exciting when all the laws of thought are made explicit human thought made explicit I

think that once you understand computational reducibility it is uh it's neither of those things because the fact is people say for example that people

will say oh but you know I have free will I I kind of um you know I operate in a way that is uh uh you know you you the the the the they have the idea that

they're doing something that is sort of of internal to them that they're figuring out what's what's happening but in fact we think there are laws of physics that ultimately determine you

know every uh every nerve you know every electrical impulse and a nerve and things like this so you might say isn't it depressing that we are ultimately just determined by the rules of physics

so to speak um it's the same thing it's at a higher level it's like it's it's it's a shorter distance to get from kind of semantic grammar to the way that we might

construct a piece of text than it is to get from Individual nerve firings to how we construct a piece of text but it's not fundamentally different and by the way as soon as we have this kind of

level of you know this other level of description it's kind of it helps us to go even further so we'll end up being able to produce more and more complicated Kinds of Kinds of things

that just like when we you know if we didn't have a computer and we knew certain rules we could write them down and go a certain distance but once we have a computer we can go vastly further and this is the same kind of thing you

wrote a blog post titled what is Chad GPT doing and where does it work we've been talking about this but can we just step back and Linger on this question what what's it what's Chad GPT doing

what are these um a bunch of billion parameters trained on a large number of words

why does it seem to work again is it is it because to the point you made that there's laws of language that can be discovered by such a process is there something well let's let's talk about

sort of the low level of what chat GPT is doing I mean ultimately you give it a prompt it's trying to work out you know what should the next word be right which

is wild isn't that isn't that surprising to you that this kind of low level dumb training procedure can create something

syntactically correct first and then semantically correct you know the thing that has been sort of a story of my life is realizing that simple rules can do much more complicated things than you

imagine that something that starts simple and start simple to describe can grow a thing that is you know vastly more located than you can imagine and

honestly it's taken me I don't know I've sort of been thinking about this now 40 years or so and it always surprises me I mean even for example in our physics project sort of thinking about the whole universe growing from these simple rules

I still resist because I keep on thinking you know how can something really complicated arise from something that simple it just seems you know it seems wrong but yet you know the

majority of my life I've kind of known from from things I've studied that this is the way things work so yes I it is wild that it's possible to write a word at a time and produce a coherent essay

for example but it's worth understanding kind of how that's working I mean it's kind of like if if it was going to say you know the cat sat on the what's the next word okay so how does it figure out

the next word well it's seen a trillion words written on the internet and it's seen the cat sat on the floor the cat sat on the sofa the cat sat on the

whatever so it's minimal thing to do is just say let's look at what we saw on the internet we saw you know 10 000 examples of the cat sat on the what was

the most probable next word let's just pick that out and say that's the next word and that's that's kind of what it is at some level is trying to do now the problem is there isn't enough text on

the internet to uh for if you have a reasonable length of prompt to that that Pro that specific prompt will never have occurred on the internet and as you as

you kind of go further there just won't be a place where you could have trained you know where you could just worked out probabilities from what was already there um you know like if you say two plus two

there'll be a zillion examples of two plus two equaling four and a very small number of examples of two plus two equals five and so on and you can pretty much know what's going to happen so then

the question is well if you can't just work out from examples what's going to happen just no probabilistic for you for example is what's going to happen you have to server model and there's kind of

an idea this idea of making models of things is an idea that really I don't know I think Galileo probably was one of the first people who sort of worked this out and it's kind of like like you know

I think I gave an example of that the book I wrote about about Chachi BT where it's kind of like you know Galileo was dropping cannonballs off the off the different floors of the of the Tower of

Pisa and it's like okay you drop a cannonball off this floor you drop a cannonball off this floor you miss floor five or something for whatever reason but you know the time it took the Cannonball to fall to the ground from

floors one two three four six seven eight for example then the question is can you work out can you can you make a model which figures out how long does it take the ball how long would it take the

ball to fall to the ground from the floor you didn't explicitly measure and the thing Galileo realized is that you can use math you can use mathematical

formulas to make a model for how long it will take the ball to fall so now the quest question is well okay you want to make a model for for example something much more elaborate like you've got this

arrangement of pixels and is this arrangement of pixels an A or a B does it correspond to something we'd recognize as an A or B and you can make a similar kind you know each pixel is

like a parameter in some equation and you could write down this giant equation where the answer is either you know a or you know one or two A or B um and the question is then what kind of

a model successfully reproduces the way that we humans would would conclude that this is an A and this is a b you know if there's a complicated extra tail on the

top of the a would we then conclude something different what is the type of model that Maps well into the way that we humans make distinctions about things and the big kind of meta Discovery is

neural Nets are such a model it's not obvious they would be such a model it could be that human distinctions are not captured you know we could try searching around for a Titan model that could be a

mathematical model it could be some model based on something else that captures kind of typical human distinctions about things it turns out this model that actually is very much

the way that we think the architecture brains works that perhaps not surprisingly that model actually corresponds to the way we make these distinctions and so you know the the

core next point is that the the kind of model that's neural net model makes sort of distinctions and generalizes things in sort of the same way that we humans

do it and that's why when you say you know the cat set on the green blank even though it never didn't see many examples of the cat set on the green whatever it

can make a or the aardvark sat on the green whatever I'm sure that particular sentence does not occur on the internet and so it has to make a model that

concludes what you know it has to kind of generalize from what it's from the actual examples that it's seen and so so you know that that's the fact is that

neural Nets generalized in the same kind of a way that we humans do if if we were you know the aliens might look at our neural net generalizations and say that's crazy you know that thing when

you put that extra little dot on the a that isn't an a anymore that's you know that messed the whole thing up but for us humans we make distinctions which seem to correspond to the kinds of

distinctions that neural Nets make so then you know the the thing that is just amazing to me about chat gbt is how similar the structure it has is to the

very original way people imagine neural Nets might work back in 1943 and you know there's a lot of detailed engineering you know great cleverness but it's really the same idea and in

fact even the sort of elaborations of that idea where people said let's put in some actual particular structure to try and make the neural net more elaborate to be very clever about it most of that

didn't matter I mean there's some things that seem to you know when you when you train this neural net you know the one thing this kind of Transformer architecture this attention idea that

really has to do with does every one of these neurons connect to every other neuron or is it somehow causally localized so to speak does it like we're making a sequence of words and the words

depend on previous words rather than just everything can depend on everything and that seems to be important in just organizing things so that you don't have a sort of a giant mess but the thing you

know the thing worth understanding about what is chat gpg in the end I mean what is a neuron that's in the end a neural net in the end is each neuron has a it it's taking inputs from a bunch of other

neurons it's it's eventually it's going to have it's going to have a a numerical value it's going to compute some number and it's it's saying I'm going to look at the the neurons above me it's kind of

a series of layers it's going to look at the neurons above me and it's going to say what are the values of all those neurons then it's going to add those up and multiply them by these weights and then it's going to apply some function

that says if it's bigger than zero or something then make it one or an otherwise make it zero or some slightly more complicated function you know very well how this works but it's a giant

equation although a lot of variables you mentioned figuring out where the ball Falls when you don't have data on the fourth floor um this the equation here is not as

simple as right the equation with 175 billion terms and it's quite surprising that in some sense a simple procedure of

uh training such an equation can lead to well I think that good representation of natural language right the the real issue is you know this architecture of a neural net where where what's happening

is you know you've you've you've turned so neural Nets always just deal with numbers and so you know you've turned the sentence that you started with into a bunch of numbers like let's say by

mapping you know each word of the 50 000 words in English you just map each word or each part of a word into some number they feed all those numbers in and then the thing is going to and then

those numbers just go into the values of these neurons and then what happens is it's just Rippling down going layer to layer until it gets to the end I think chat gpg has about 400 layers and you're

just you know it just goes once through it just every every new word it's going to compute just says here are the here are the numbers from the words before let's compute the what is it compute it

computes the probabilities that it estimates for each of the possible 50 000 words that could come next and then it decides sometimes it will use the most probable word sometimes it will use not the most probable word it's an

interesting fact that there's this so-called temperature parameter which you know at temperature zero it's always using the most probable word that it can that it estimated was the the most probable thing to come next you know if

you increase the temperature it'll be more and more kind of random in its selection of words it'll go down to lower and lower probability words thing I was just playing with actually recently was the transition that happens

as you increase the temperature the thing goes Bonkers at a particular you know sometimes at a particular temperature I think maybe about 1.2 is the thing I was noticing from yesterday actually

um that you know usually it's giving reasonable answers and then uh at that temperature with some probability it just starts spouting nonsense

um and you know nobody knows why this happens I mean it's it's uh uh and by the way I mean the thing to understand is it's putting down one word at a time

but the outer loop of the fact that it says okay I put down a word now let's take the whole thing I wrote so far let's feed that back in let's put down another word that outer loop which seems

almost trivial is really important to the operation of the thing and and for example one of the things that is kind of funky is it'll give an answer and you say to it is that answer correct and

it'll say no and why is that happening right right why couldn't it do that well the answer is because it is going one word at a time sort of forwards and it didn't you

know it it came along with some sort of chain of of thought in a sense and it came up with completely the wrong answer but as soon as you feed it the whole thing that it came up with it

immediately knows that that isn't right it immediately can recognize that was a you know a bad syllogism or something and uh can see what happened even though as it was being led down this Garden

Path so to speak it didn't it came to the wrong place but it's fascinating that this kind of procedure converges to something that forms a pretty good

compressed representation of language on the internet yeah that's quite right right no I'm not sure what to make of it well look I think you know there are

many things we don't understand okay so for example you know 175 billion weights it's maybe about a trillion bytes of information which is very comparable to

the training set that was used um and uh you know why that why kind of it sort of stands to some kind of reason that the number of Weights in the neural net I don't know where I can't really

argue that I can't really give you a good uh you know in a sense the very fact that you know the insofar as there are definite rules of what's going on you might expect that eventually we'll

have a much smaller neural net that will successfully capture what's happening I I don't think the best way to do it is probably a neural net I think a neuron that is what you do when you don't know any other way to structure the thing and

it's a very good thing to do if you don't know any other way to structure the thing and for the last 2000 years we haven't known any other way to structure it so this is a pretty good way to start but that doesn't mean you can't find

sort of in a sense more symbolic rules for what's going on that you know much of which will then be you can kind of get rid of much of the structure of the neural Nets and replace it by things

which are sort of pure steps of computation so to speak sort of with neural net stuff around the edges and that becomes just a you know just a much simpler way to do it so then you're on

that you hope will reveal to us good symbolic rules that make the need then you're on that less and less and less right and and there will still

be some stuff that's kind of fuzzy just like you know that they're things that it's like this question of what can we formalize what can we turn into computational language what is just sort

of oh it happens that way just because brains are set up that way what do you think are the limitations of uh large language models just to make it

explicit well I mean I think that deep computation is not what large language models do I mean that's just it's a different kind of thing you know the outer loop of a large language model if if you're trying to do many steps in a

computation the only way you get to do that right now is by spooling out you know all that the whole Chain of Thought as a bunch of words basically and you know you can make a turing machine out

of that if you want to I just was make doing that construction you know in principle you can make an arbitrary computation by just spooling out the words but it's an it's a bizarre and

inefficient way to do it um but it's something where uh the you know I I think that's you know sort of

the the Deep computation is it's it's really what a humans can do quickly large language models will probably be able to do well anything that you can do

kind of off the top of your head type thing is is really you know is good for large language models and the things you do off the top of your head you may not get them always right but you know

you'll it it's it's thinking it through the same way we do but I wonder if there's an automated way to do something that humans do well much faster to where it like Loops

so generate arbitrary large code bases of Wolfram language for example well the question is what does what do you want the code base to do

um Escape control and take over the world okay so you know the thing is when people say you know we want to build

this giant thing right a giant piece of computational language in a sense it's sort of a failure of computational language if the thing you have to build in other words if we have a description

if you have a small description that's the thing that you represent in computational language and then the computer can compute from that yes so in

a sense in you know when as soon as you're giving a description that you know if you have to somehow make that description something you know definite

something formal and once and and to say to say okay I'm going to give this piece of natural language and then it's going to splurt out this giant formal

structure that in a sense that doesn't that doesn't really make sense because acceptanceofar as that piece of natural language kind of plugs into what we

socially know so to speak it plugs into kind of our Corpus of knowledge then you know that's the way we're capturing a piece of that Corpus of knowledge but hopefully we will have done that in computational language how do you make

it do something that's big well you know you have to have a way to describe what you want okay I can make it more explicit if you want how about I just pop into my head

um iterate through all the members of Congress and figure out how to convince them that they have to let

me this meaning the system become president pass all the laws that allows AI systems to take control and be the president I don't know so that's a very explicit

like figure out the individual life story of each Congressman that each Senator anybody I don't know what's required to really kind of pass legislation and figure out how to

control them and manipulate them get all the information what would be the biggest fear of this congressman and uh in such a way that you can take action

on it in the digital space so maybe threaten the destruction reputation or something like this right if I can describe what I want yeah you know to what extent can a large language model

automate that with the help uh with the help of the concretization of something like Wolfram language that makes it more

um yeah rather a long way I'm also surprised how quickly I was able to generate yeah yeah right an attack that that's the here you know I I swear

I swear I did not think about this before it is funny how quickly which is a very concerning thing because that that probably this idea will probably do quite a bit of damage and there might be a very large number of other such ideas

well I'll give you a much more benign version of that idea okay you're going to make an AI tutoring system and you know that is a that's a benign version of what you're saying is

um I want this person to understand this point yes you know you're essentially doing machine learning where the where the where the you know the the loss function the the thing you're trying to get to is get the human to understand

this point and and when you do a test on the human that they yes they correctly understand how this or that works and I I am confident that uh you know sort of

the large language model type technology combined with computational language is going to be able to do pretty pretty well at teaching us humans things and it's going to be an interesting

phenomenon because you know sort of individualized teaching is is a thing that has been kind of a you know a goal for a long time I think we're going to get that I think more you know that it

has many consequences for you know like like just you know if you know me as an if you the AI know me tell me I'm about to do this thing what is the what are

the three things I need to know you know given what I already know you know what's the what's let's say I'm I'm looking at some paper or something right

it's it's like there's a version of this summary of that paper that is optimized for me so to speak and where it really is and I think that's really going to work it could understand the the major

gaps in your knowledge yes that if field would actually give you uh a deeper understanding of the topic here right and that's a you know that's an important thing because it really

changes actually I think you know when when you think about education and so on it really changes kind of what's worth doing what's not worth doing and so on it makes you know I know in my life I've

learned lots of different fields and you know so I yeah I don't know I have every time I'm always think that this is the one that's going to I'm not going to be able to learn yeah but turns out sort of there are sort of meta methods for

learning these things in the end um and you know I think this this idea that it becomes easier to you know it becomes easier to be fed knowledge so to

speak and it becomes you know if you need to know this particular thing you can you know you can get taught it in an efficient way it's something I think is

sort of a an interesting feature and I think it makes the um you know things like the value of of big towers of specialized knowledge become

less significant compared to the kind of meta knowledge of sort of understanding kind of the the big picture and being able to connect things together I think that you know there's been this huge

trend of let's be more and more specialized because we have to you know we we have to sort of Ascend these towers of knowledge but by the time you can get you know more automation of

being able to get to that place on the tower without having to go through all those steps I think it sort of changes that picture interesting so your intuition is that in terms of the the

the collective intelligence of the species in the individual minds they make up that Collective there'll be more there will Trend

towards being generalists and being kind of philosophers that's what I think I think that's where the humans are going to be useful I think that a lot of these kind of

the drilling the the the mechanical working out of things is much more automatable it's much more ai ai territory so to speak no more phds

well that's a it's interesting yes I mean that you know the the kind of the specialization this kind of tower of specialization which has been a feature of you know we've accumulated lots of

knowledge in our in our species and and you know in a sense every time we every time we have a kind of automation a building of tools it becomes less

necessary to know that whole Tower and it becomes something where you can just use a tool to get to the top of that Tower I I think that um uh you know the thing that is ultimately you know when

we think about okay what do the AIS do versus what do the humans do it's like ai's you tell them you say go achieve this particular objective okay they can maybe figure out a way to achieve that

objective we say what objective would you like to achieve the AI has no intrinsic idea of that it's not a defined thing that's a thing

which has to come from some other you know some other entity and insofar as we are in charge so to speak or whatever it is and our kind of web of society and

history and so on is the thing that is defining what objective we want to go to that's you know that that's that's a thing that we humans are necessarily

involved in so to push back a little bit don't you think that GPT feature versions of GPT would be able to give a good answer to

what objective would you like to achieve from on what basis I mean if they say look here's the terrible thing that could happen Okay that taking the average of the internet and they're

saying you know from the average of the internet what do people want to do well that's the uh the Elon Musk outage of the most entertaining outcome is the most likely

okay that could be um that could be one objective is maximize global

entertainment the dark version of that is drama the the good version of that is fun right so I mean this this question of

what uh you know if you say to the AI um you know uh what does the species want to achieve yes okay there'll be an answer right

that'll be an answer it'll be what the average of the internet says the species wants to achieve well this this I think you're using the word average very Loosely there right so

I think you I think the answers will become more and more interesting as these language models are trained better and better no but I mean in the end it's a reflection back of what we've already

said yes but it's uh there's a deeper wisdom to the collective intelligence presumably than each individual maybe isn't that what we're trying to just

Society uh to have well I mean that's that's a that's an important no that's an interesting question I mean in you know insofar as some of us you know work

on trying to innovate and figure out new things and so on it is sometimes it's a it's a complicated interplay between sort of the individual doing the crazy

thing often some some spur so to speak versus the collective that's trying to do sort of the the the the high inertia average thing and it's you know

sometimes the collective you know is is bubbling up things that are interesting and sometimes it's pulling down kind of the attempt to make this kind of innovative Direction but don't you think

the large language models would see beyond that simplification will say maybe intellectual and career diversity is really important so you need the crazy people from the outlier on the

outskirts and so like the actual what's the purpose of this whole thing is to explore through this kind of dynamics that we've been using as a human civilization which

is most of us focused on one thing and then there's the crazy people on the outskirts doing the opposite of that one thing and you kind of pull the whole society together there's the mainstream science and then there's the crazy

science and it's just been about the history of human civilization and maybe the AI system will be able to see that and the more and more impressed we are by a language model telling us this the

more control we'll give it to it and the more we'll be willing to let it run our society and hence there's this kind of loop where the society could be

manipulated to let the AI system run it right well I mean look one of the things that sort of interesting is we might say we always think we're making progress

but yet if you know in a sense by by saying let's take what already exists and use that as a model for what should exist yeah then you know it's

interesting that for example you know many religions have taken that point of view there is a you know a sacred book that got written at Timex and it defines how people should act for all future

time and that's you know it's it's a model that people have operated with and in a sense you know this is a version of that that kind of statement it's like

take the 2023 version of sort of how the world has exposed itself and use that to Define what the world should do in the future but it's not it's an imprecise definition right

because just like with religious text and with GPT the human interpretation of what GPT says will be the um

uh will be the perturbation in the system it'll be the noise it'd be full of uncertainty it's not like GPT will tell you exactly what to do

it'll tell you approx A Narrative of what like uh uh you know it's like turn the other cheek kind of narrative right that's that's not a fully instructive narrative well until until the AIS

control all the systems in the world they will be able to very precisely tell you what to do but they'll do what they you know they'll they'll just do this or that thing and and that and and not only

that they'll be Auto suggesting to each person you know do this next do that next so I think it's a it's a slightly more prescriptive situation than one has

typically seen but you know I think this this whole question of sort of what what's left for the human so to speak to

what extent do we uh you know this idea that there is an existing kind of Corpus of purpose for humans defined by what's on the internet and so on that's an

important thing but then the question of sort of as we explore what we can think of as the computational universe as we explore all these different possibilities for what we could do all these different inventions we could make

all these different things the question is which ones do we choose to follow those choices are the things that in a

sense if the humans want to still have kind of human progress that's what we we get to make those choices so to speak in

other words the the there's this idea if you say let's take the uh kind of what exists today and use that as the determiner of all of what

there is in the future the thing that is sort of the opportunity for humans is there will be many possibilities thrown up there are many different things that could happen

or be done and the insofar as we want to be in the loop the thing that makes sense for us to be in the loop doing is picking which of those possibilities we want

but the degree to which there's a feedback loop of the idea that we're picking something starts becoming questionable because we're influenced by the various systems

absolutely like if that becomes more and more source of our education and wisdom and knowledge right the AIS take over I mean my you know I've thought for a long time that

you know it's the you know AR Auto suggestion that's really the thing that makes the AIS take over it's just then the humans just follow you know yeah we will no longer write emails to each

other we'll just send the auto suggested email yeah yeah but the the thing where humans are potentially in the loop is when there's a choice and when there's a

choice which we could make based on our kind of whole web of history and so on yeah and and that's you know that's insofar as it's all just you know

determined uh you know the humans don't have a place and and by the way I mean you know at some level uh you know it's all kind of a complicated philosophical

issue because at some level the universe is just doing what it does we are parts of that universe that are necessarily doing what we do so to speak yet we feel

we have sort of agency in what we doing and that's that's its own separate kind of interesting issue and we also kind of feel like we're the Final Destination

what the universe was meant to create uh but we very well could be and likely are some kind of intermediate step

obviously yeah well we're we're most certainly some intermediate step the question is if there's some cooler more complex more interesting uh thing that's going to be materialized this

computational universe is full of such things but in our particular pocket specifically if this is the best we're going to do or not that's kind of a we can make all kinds of interesting things

in the computational universe we when we look at them we say yeah you know that's that's a thing we don't it doesn't really connect with our current uh our current way of thinking about things

it's like in mathematics you know we've got certain theorems they're about three or four million that human mathematicians have written down and published and so on but they're an infinite number of possible mathematical theorems we just go out into the

universe of possible theorems and pick another theorem and then people will say well you know that's the you know they look at it and they say I don't know what this theorem means it's not

connected to the things that are part of kind of the web of history that we're dealing with you know I think one one point to make about sort of understanding Ai and its relationship to us is as we have this kind of whole

infrastructure of AIS doing their thing and doing their thing in a way that is perhaps not readily understandable by us humans you know you might say that's a that's a very weird situation how can we

have built this thing that behaves in a way that we can't understand that's full of computational irreducibility Etc et cetera et cetera you know what what is this what's it going to feel like when

the world is run by AIS whose operations we can't understand and the thing one realizes is actually we've seen this before that's what happens when we exist in the natural world the natural world

is full of things that operate according to definite rules they have all kinds of you know computational irreducibility we don't understand what the natural world is doing occasionally and you know when

you say you know are the AIS going to wipe us out for example well it's kind of like is the machination of the AIS going to lead to this thing that eventually comes and destroys the species well we can also ask the same

thing about the natural world or the machination of the natural world going to eventually lead to this thing that's going to you know make make the earth explode or something like this those are

those are questions those are and insofar as we think we understand what's happening in the natural world that's a result of Science and natural science and so on one of the things we can expect when there's this giant

infrastructure of the AIS is that's where we have to kind of invent a new kind of natural science that kind of is the natural science that explains to us how the AIS work it's kind of like we

can we can you know we have a I don't know a horse or something and we're trying to get it we're trying to you know ride the horse and go from here to there we don't really understand how the horse Works inside but we can get

certain rules and certain you know approaches that we take to persuade the horse to go from here to there and and and take us there and that's the same type of thing that we're kind of dealing

with with the sort of incomprehensible computationally irreducible AIS but we can identify these kinds of we can find these kind of pockets of reducibility

that we can kind of uh you know the I don't know we're grabbing onto the main of the horse or something to be able to to write it um or we figure out you know if we if we

do this or that to to ride the horse that that's a a successful way to to get it to do what what we're interested in doing there does seem to be a difference between a horse

and a um a large language model or something that could be called AGI connected to the internet so let me just ask you about big philosophical question

about the threats of these things there's a lot of people like Eliezer adkowski who worry about the existential risks of AI systems is that something

that you worry about you know sometimes when you're building an incredible system like wolf from alpha you can kind of get lost in it I like try and think a little bit about the

implications of what one's doing you know it's like the Manhattan Project kind of situation where you're like it's some of the most incredible physics in engineering being done but it's like huh where's this gonna go I think some of

these arguments about kind of you know they'll always be a smarter AI they'll always be you know and eventually the AIS will get smarter than us and then all sorts of terrible things will happen

to me some of those arguments remind me of kind of the ontological arguments for the essence of God and things like this they're kind of arguments that are based on some particular model fairly simple

model often of kind of there is always a greater this that and the other you know this is um and that's you know those arguments ten what tends to happen in the sort of reality of how these things develop is that it's more complicated

than you expect that the kind of simple logical argument that says oh eventually there'll be a super intelligence and then it will you know do this and that turns out not to really be the story it

turns out to be complicated story so for example here's an example of an issue is there an apex intelligence just like there might be an apex predator in some you know ecosystem is there going to be

an apex intelligence the most intelligent thing that there could possibly be right I think the answer is no and in fact we already know this and it's a kind of a back to the whole

computational reducibility story there's kind of a question of you know even if you have um if you if you have sort of a uh a turing

machine and you have a turing machine that that runs as long as possible before before it halts you say is this the machine is this the Apex machine that does that there will always be a

machine that can go longer and as you go out to the infinite collection of possible Turing machines you'll never have reached the end so to speak you'll never you'll always be able to it's kind of like the same same question of whether there'll always be another

invention will you always be able to invent another thing the answer is yes there's an infinite Tower of possible inventions that's one definition of apex uh but the other is like

which I also thought you were which I also think might be true is is there a species that's the Apex intelligence right now on Earth so it's not trivial to say that humans

are that yeah it's not trivial I agree it's a you know I think one of the things that I I've long been curious about kind of other intelligences so to speak

um I mean I you know I I view intelligence is like computation and it's kind of a you know you're sort of you have the set of rules you deduce

what happens um I have tended to think now that there's this kind of specialization of computation that is sort of a consciousness-like thing that has to do

with these you know computational boundedness single thread of experience these kinds of things that are the specialization of computation that corresponds to a somewhat human-like

experience of the world now the question is so so that's you know there may be other intelligences like you know you know the aphorism you know the weather has a mind of its own it's a different

kind of intelligence that can compute all kinds of of things that are hard for us to compute but it is not well aligned with us with the way that we think about things it doesn't it doesn't it doesn't

think the way we think about things and you know in this idea of different different intelligences every different mind every different human mind is a different intelligence that thinks about

things in different ways and you know in in terms of the kind of formalism of our physics project we talk about this idea of rule space the space of all possible sort of rule systems and different minds

are in a sense of different points in real space human Minds ones that have grown up with the same kind of culture and ideas and things like this might be pretty close in real space pretty easy

for them to communicate pretty easy to translate pretty easy to move from one place in rural space that corresponds to one mind to another place in rule your space that corresponds to another sort

of nearby mind when we deal with kind of more distant things in rural space like you know the the pet cat or something um you know the pet cat has some aspects

that are shared with us the emotional responses of the cat are somewhat similar to ours but the cat is further away in real space than people are and

so then the question is you know can we identify sort of the can we make a translation from our thought processes to the thought process of a cat or something like this and you know what

what will we get when we you know what what will happen when we get there and I think it's the case that that many you know many animals I don't know dogs for example you know they have a labradal

factory systems they you know they they have sort of the smell architecture of the of the of the world so to speak in a way that we don't and so you know if if

you were sort of talking to the dog and you could you know communicate in a language the dog will say well this is a you know a a you know a flowing smelling

this that and the other thing Concepts that we just don't have any idea about now what's what's interesting about that is one day we will have chemical sensors

that do a really pretty good job you know we'll have artificial noses that work pretty well and we might have our augmented reality system show us kind of the same map that the dog could see and things like this so that you know

similar to what happens in the dog's brain and eventually we will have kind of expanded in real space to the point where we will have those same sensory experiences that dogs have and we will

have internalized what it means to have you know the smell landscape or whatever and and so then we will have kind of colonized that part of Royal space

um until you know we haven't gone you know some things that that you know animals and so on do we've sort of successfully understand others we do not

and the question of of what kind of what is the uh you know what what representation you know how how do we convert things that animals think about

to things that we can think about that's not a trivial thing um and you know I've I've long been curious I've had a very bizarre project at one point of of trying to make an

iPad game that a cat could win against its owner right it feels like there's a deep philosophical go there though yes yes I mean the the

you know I was curious if you know if pets can work in Minecraft or something and can construct things what will they construct and will what they construct be something where we look at and we say

yeah I recognize that or will it be something that looks to us like something that's out there in the computational universe that one of my you know cellular automata might have produced where we sell yeah I can kind

of see it operates good into some rules I don't know why you would use those rules I don't know why you would care yeah I uh actually it's just a link on that seriously is there a connector in

the Royal space between you and a cat where the cat could legitimately win so iPad is a very limited um interface yeah I I wonder if there's

a game where cats win I think the problem is the cats don't tend to be that interested in what's happening on the iPad yeah that's an interface issue probably yeah right right no I I think

it is likely that I mean you know there are plenty of animals that would successfully eat us if we were you know if we were exposed to them and so there's you know it it's gonna pounce

faster than we can get out of the way and so on so there are plenty of and and probably it's going to you know we think we've hidden ourselves but we haven't successfully hidden ourselves that's a

physical strength I wonder if there's uh something in more in the realm of uh intelligence where an animal like a cat could out well I think there are things certainly in terms of the the speed of

processing certain kinds of things for sure I mean the the question of what you know is there a game of chess for example is there cat chess yeah that the cats could play against each other and

if we tried to play a cat we don't always lose I don't know you might have to do with speed but it might have to do with Concepts also there might be

Concepts and the cats had I I tend to think that our species from its invention of language has managed to build up this kind of tower of

abstraction that for things like a chess-like game will make us win in other words we've become through the fact that we've kind of experienced language and learned abstraction you

know we've sort of become smarter at those kinds of abstract kinds of things now you know that doesn't make us smarter at catching a mouse or something it makes us smarter at the things that

we've chosen to to sort of con you know concern ourselves which are these kind of abstract things yeah and I think you know this is again back to the question of of you know what does one care about

you know if one's a if one's the you know the cat if you if you have the discussion with a cat if we can if we can translate things to have the discussion with a cat the cat will say I

you know I'm very excited that uh this light is moving and we'll say why do you care and the cat will say that's the most important thing in the world yeah that

this thing moves around I mean it's like when you ask about I don't know you you look at archaeological remains and you say these people had this you know belief system about this and you know that was the most important thing in the

world to them um and and now we look at it and say we don't know what the point is but it was I mean I I've been curious you know that these hand prints on caves from 20 000 or more years ago and it's like nobody

knows what these handprints were there for you know that they may have been a representation of the most important thing you can imagine they may just have been some you know some kid who who

rubbed their hands in the mud and stuck them on the walls of the cave you know we don't we don't know and I think but this this whole question of what um you know is

when you say uh this question of sort of what's the smartest thing around there's the question of what kind of computation are you trying to do if you're saying

you know if you say you've got some well-defined computation and how do you implement it well you could implement it by nerve cells you know firing you can implement it with silicon and

electronics you can implement it by some kind of molecular computation process in the human immune system or in some molecular biology kind of thing they're different ways to implement it and you

know I think this question of of uh of sort of which you know those different implementation methods will be of different speeds they'll be able to do different things if you say uh you know

which so an interesting question would be um what kinds of abstractions are most natural in these different kinds of

systems so for a cat it's for example you know the visual scene that we see you might you know we pick out certain objects we recognize you know certain

things in that visual scene a cat might in principle recognize different things I I suspect you know Evolution biological evolution is very slow and I suspect what a cat notices is very

similar we even know that from some neurophysiology what a catnosis is is very similar to what we notice of course there's a you know one obvious differences cats have only two kinds of

color receptors so they don't see in the same kind of color that we do now you know we say we're we're better we have three color receptors you know red green blue we're not the overall winner I

think the the I think the mantis shrimp is the overall winner winner with 15 color receptors I think so it can it can kind of make distinctions that with our

current you know like the Mantis shrimp's view of reality is inside at least in terms of color is much richer than ours um now but what's interesting is how do

we get there so imagine we have this augmented reality system that is even you know it's singing into the infrared into the ultraviolet things like this and it's Translating that into something

that is connectable to our brains either through our eyes or more directly into our brains um you know then eventually our kind of web of the types of things we understand

will extend to those kinds of constructs just as they have extended I mean there are plenty of things where we see them in the modern world because we made them with technology and now we understand

what that is but if we'd never seen that kind of thing we wouldn't have a way to describe it we wouldn't have a way to understand it and so on all right so that actually stemmed from our

conversation about whether AI is going to kill all of us and you we've discussed this kind of spreading of intelligence through rule

space that in practice it just seems that things get more complicated things are more complicated than the story of well if you build a thing that's plus

one intelligence that thing will be able to build the thing that's plus two intelligence and plus three intelligence and that will be exponential it'll become uh more intelligent exponentially

faster and so on until it completely destroys everything um but you know that intuition might still not be so simple but I might still

care carry validity and there's two interesting trajectories here one a super intelligence system remains in rulio

proximity to humans to where we're like holy crap this thing is really intelligent uh let's select the present and then there could be perhaps more

terrifying intelligence that starts moving away they might be around us now they're moving far away in rural space but they're still sharing physical

Resources with us yes yes so they can rob us of those physical resources and Destroy Humans just kind of casually yeah just just uh like nature code like

nature could but it seems like there's something unique about AI systems where um there is a this kind of exponential

growth like the way well sorry Nature has so many things in it one of the things that nature has which is very interesting are viruses for example there is systems within nature that have

this kind of exponential effect and that terrifies us humans because it can you know there's only eight billion of us and you can just kind of it's not that hard to just kind of whack

them all real quick so uh I mean is that something you think about that yeah I thought about that yes the threat of it yeah you as concerned about it as uh

somebody like Elias yakovski for example just big big painful negative effects of AI about Society you know no but perhaps that's because

I'm intrinsically an optimist um I mean I think that there are things I think the thing that one you know one

sees is there's going to be this one thing and it's going to just zap everything somehow you know I maybe I have faith in computational irreducibility so to speak that there's

always unintended little Corners that you know it's just like somebody says I'm going to oh I don't know somebody has some some bio weapon and they say we're going to release this and it's going to do all this harm but then it

turns out it's more complicated than that because you know the kind of some humans are different and you know the exact way it works is a little different than you expect it's something where

sort of the the the great big you you know you smash the thing with something you know you the asteroid collides with the Earth yeah and it kind of you know and yes you know the Earth is cold for

two years or something and you know then lots of things die but not everything dies and it's you know there's there's usually I mean I I kind of this is in a sense the sort of story of computational irreducibility there are always

unexpected Corners there always unexpected consequences and I don't think that they're kind of whack it over the head with something and then it's all gone is you know that can obviously happen the Earth can be swallowed up in

a black hole or something and then it's kind of presumably presumably all over um the uh but but you know I think this question of of what you know what what

do I think the realistic paths are I think that there will be sort of an increasing I mean the the people have to get used to phenomena like computational

irreducibility there's an idea that we built the machines so we can understand what they do and we're we're going to be able to control what happens well that's

not really right now the question is is the result of that lack of control going to be that the machine's kind of conspire and sort of wipe us out maybe

just because I'm an optimist I don't tend to think that that's you know that's in the cards I think that the you know as a realistic thing I suspect you

know what will sort of emerge maybe is kind of an ecosystem of the AIS just as you know again I I don't really know I mean this is something that's it's hard to it's hard to be clear about what will

happen um I mean I think that the you know there are there are a lot of sort of details of you know what could we do what systems in the world could we connect and AI to you know I have to say

I was just a couple of days ago I was working on this uh chat gbt plug-in kit that we have for open language okay where you can you know you can create a

plugin and it runs with language code um and it can run more from language code back on your own computer yeah and I was thinking well I can just make it you know I can tell chat gbt create a

piece of code and then just run it on my computer and I'm like you know that that sort of personalizes for me the what could what could possibly go wrong so to speak was that exciting or scary that possibility

it was a little bit scary actually because it's kind of like like I realize I'm I'm delegating to the AI just write a piece of code you know you're in charge write a piece of code run it on my computer

and pretty soon all my files exactly let's take a uh that's like Russian Roulette but like much more complicated yeah yes yes right that's a good

drinking game I don't know so well uh right I mean it's an interesting question then if you do that right what is the sandboxing that you should have and that's sort of

a that's a a version of of that question for the world that is as soon as you put the AIS in charge of things you know how much how many constraints should there be on these systems before you put the

AIS in charge of all the weapons and all these you know all these different kinds of systems well here's the fun part about sandboxes is uh the AI knows about them it has the tools to uh crack them

look the fundamental problem of computer security is computational irreducibility yes because the fact is any sandbox is never any you know it's never going to

be a perfect sandbox if you want the system to be able to do interesting things I mean this this is the the problem that's happened the generic problem of computer security that as soon as you have your you know firewall

that is sophisticated enough to be a universal computer that means it can do anything and so long as if you find a way to poke it so that you actually get it to do that Universal computation

thing that's the way you kind of crawl around and get her to do the thing that it wasn't intended to do and that's sort of a another version of computational irreducibility is you can you know you

can kind of you get it to do the thing you didn't expect it to do so to speak there's so many interesting possibilities here that manifest themselves from the compute computationally reducibility here

it's just so many things could happen because in digital space things move so quickly you can have a chat bot you can have a piece of code that you could basically have chat GPT

generate viruses accidentally or on purpose and they uh digital viruses yes and uh they could be brain viruses too they they convince kind of like uh

phishing emails yes they can convince you of stuff yes and no doubt you can you know in a sense we've had the loop of the machine learning Loop of making things that convince people of things is

surely going to get easier to do yeah and you know then what does that look like well it's again you know we humans are you know we're this is a new

environment for us and admittedly it's an environment which a little bit scarily is is changing much more rapidly but than I mean you know people worry about you know climate change is going

to happen over hundreds of years and you know the environment is changing but the environment for you know in the the kind of digital environment might change in in six months

so one of uh the relevant concerns here in terms of the uh impact of GPT on society is the nature of Truth that's relevant to Wolfram Alpha because

computation through symbolic reasoning that's embodied in Wolfram Alpha is the interface there's a kind of sense that

what Wolfram Alpha tells me is true so we hope yeah I mean you could probably analyze that you could show you can't prove that it's always going

to be true computational disability uh but it's gonna be more true than not it's look the fact is it will be the correct consequence of the

rules you've specified and insofar as it talks about the real world you know that is our job in sort of curating and collecting data to make sure that that

data is quotes as true as possible now what does that mean well you know it's always an interesting question I mean for us our operational definition of truth is

you know somebody says who's the best actress who knows but somebody won the Oscar and that's a definite fact yeah and so you know that's the kind of thing that we

can make computational as a piece of truth if you ask you know these things which you know a sensor measured this thing it did it this way a machine Learning System this particular machine

Learning System recognized this thing that's a that's a sort of a definite uh effect so to speak and that's uh you know there are there is a good network

of those things in the world it's certainly the case that uh particularly when you say is so and so a good person you know that's a that's a hopelessly you know we might have a computational

language definition of good I don't think it'd be very interesting because that's a very messy kind of concept not really amenable to uh kind of you know that I think as far as we will get with

those kinds of things is I want X there's a kind of meaningful calculus of I want X um and that has various consequences I mean I'm not sure I haven't I haven't thought this through properly but I

think you know a concept like is so and so a good person is that true or not that's a mess that's a mess that's amenable to computation I think I think

it's a mess when humans try to Define what's good uh like through legislation but when humans try to Define what's good through literature through uh history books through poetry

it starts well I don't know I mean that particular thing it's kind of like you know we're we're going into kind of the ethics of what what counts as good so to

speak and you know what do we think is right and so on and I think that's a thing which you know one feature is uh we don't all agree about that there's no

theorems about kind of uh you know there's no there's no theoretical framework that says this is this is the way that ethics has to be well first of all there's stuff we kind of agree on

and there's some empirical backing for what works and what doesn't from just even the morals and ethics within religious texts so we seem to mostly

agree that murder is bad the certain universals that seem to emerge I wonder where the murder of an AI is bad well I tend to think yes but uh and I

think we're gonna have to contend with that question oh and I wonder what AI would say yeah well I think you know one of the things

with with AIS is it's one thing to wipe out that AI that is only you know it has no owner you can even easily imagine an AI kind of hanging out on the on the you

know on on the internet without having any particular owner or anything like that um and then you say well well what harm does it you know it's it's okay to get

rid of that AI because of the AI has 10 000 friends who are humans and all those you know all those ten thousand humans will be incredibly upset that this AI just got exterminated it becomes a

slightly different more entangled story but yeah I know I think that this question about what do humans agree about it's uh you know there are certain

there's certain things that you know human laws have tended to uh consistently agree about um you know there have been times in history when people have sort of gone

away from certain kinds of laws even ones that we would now say how could you possibly have not not done it that way um you know that just doesn't seem right at all um but I think I mean this question of

what I don't think one can say Beyond saying if you have a set of rules that will cause the species to go extinct

that's probably you know you could say that's probably not a winning set of laws because even to have a thing on which you can operate laws requires that the species not be extinct but between

sort of what's the distance between Chicago and New York that wolf from alpha can answer and the question of if this person is good or not there seems to be a lot of gray area

and that starts becoming really interesting I think you're uh since the creation of Wolfram Alpha have been a kind of arbiter of Truth at a large scale so this system is

generates more truth and try to make sure that the things are true I mean look it's a practical matter when people write computational contracts and it's kind of like you know if this happens in

the world then do this yes and this hasn't developed as as quickly as it might have done you know this has been a sort of a blockchain story in part and so on although blockchain is not really necessary for the idea of computational

contracts but you can imagine that eventually sort of a large part of what's in the world are these giant chains and networks of computational contracts and then something happens in

the world and this whole giant domino effect of contracts firing autonomously that cause other things to happen and you know for us you know we've been the

main sort of source the Oracle of of quotes facts or truth or something for things like blockchain computational contracts and such much like and there's a question of you know what you know I

consider that responsibility to actually get the stuff right and one of the things that is tricky sometimes is when is it true when is it a fact when is it

not a fact yes I think the best we can do is to say uh you know we we have a procedure we follow the procedure we

might get it wrong but at least we won't be corrupt about getting it wrong so to speak so that's beautifully put and have a transparency about the procedure foreign

the problem starts to emerge when the things that you convert into computational language start to expand for example into the realm of politics so this is where it's almost like this

nice dance of wolf from Alpha and uh Chad gbt Chad gbt like you said is a shallow and Broad so it's going to give

you an opinion on everything but it writes fiction as well as fact which is exactly how it's built I mean that's exactly it is making language and it is

making both even in code it writes fiction I mean it's kind of fun to see sometimes you know it'll write fictional World from language code yeah that that term it kind of looks right yeah it

looks right but it's actually not pragmatically correct yeah um but but yes it's it's a it has a view of kind of roughly how the world works at the same

level as as books of fiction talk about roughly how the world Works they just don't have happened to be the way the world actually worked or whatever but yes that that's um no I I agree that's

sort of a um you know we are attempting with with our whole you know wolfen language computational language thing to represent uh at least well it's either

it doesn't necessarily have to be how the actual World works because we can invent a set of rules that aren't the way the actual World works and run those rules but then we're saying we're going

to accurately represent the results of running those rules which might or might not be the actual rules of the world but we also try to capture features of the

world uh as accurately as possible to represent what happens in the world now again as we've discussed you know the the atoms in the world arranged you know

you say I don't know you know was there a tank that showed up you know that that you know drove somewhere okay well you

know what is a tank it's in a management of atoms that we abstractly describe as a tank and you could say well you know there's some arrangement of atoms that

is a different derangement of atoms but it's and it's not you know we didn't we didn't decide it's like this Observer Theory question of you know what what arrangement of atoms counts as a tank

versus not a tank so there's there's even things that would consider strong facts you could start to kind of disassemble them and show that they're not right I

mean so so the question of whether oh I don't know was this gust of wind strong enough to blow over this particular thing well a gust of wind is a complicated concept

you know it's full of little pieces of fluid dynamics and little vortices here and there and you have to Define you know was it you know what the aspect of the gust of wind that you care about

might be it put this amount of pressure on this you know blade of some some you know wind turbine or something um and uh you know that that's the um um

and but but you know if you say if you have something which is the fact of the gust of wind was the strong or whatever that you know that is you have to have some definition of that you have to have

some measuring device that says according to my measuring device that was constructed this way the Guster wind was this so what can you say about the nature of truth that's useful for us to

understand chat GPT because you've been con you've been contending with this idea of what is fact and not and it seems like Chachi Patrice used a lot now I've seen

it used by journalists to write articles and so you have um people that are working with large language models trying to desperately figure out how do we

essentially censor them through different mechanisms either manually or through reinforce and learning with human feedback try to align them to to

not say fiction just to say non-fiction as much as possible this is the importance of computational language as an intermediate it's kind of like you've got the large language model it's able

to suffice something which is a formal precise thing that you can then look at and you can run tests on it and you can do all kinds of things it's always going to work the same way and it's precisely

defined what it does and then the large language model is the interface I mean the way I view these large language models one of their important I mean there are many use cases and you know it's a remarkable thing could talk about some of these you know literally you

know every day we're coming up with a couple of new use cases um some of which are very very very surprising um and things where I mean but the best

use cases are ones where it's you know even if it gets it roughly right it's still a huge win like a use case we had from a week or two ago is read our bug reports you know we've got hundreds of

thousands of bug reports that will be accumulated over decades and it's like you know can we have it just read the bug report figure out where the where is

the bug likely to be and you know home in on that piece of code maybe they'll even suggest some some you know sort of way to fix the code it might get that it might be nonsense what it says to about

how to fix the code but it's incredibly useful that it was able to you know so awesome it's so awesome because even the nonsense will somehow be instructive I don't I don't quite understand that yet

I've yeah there's so many programming related things like uh for example uh translating for one programming language to another is really really interesting

it's extremely effective but then you the failures reveal the path forward also yeah but I think I mean the the big thing I mean in that kind of discussion the unique thing

about our computational language is it was intended to be read by humans yes and so it's really important right and so it has this thing where you can but but you know thinking about sort of church EBT and its use and so on the one

of the big things about it I think is it's a linguistic user interface that is so a typical use case might be in the take the journalist case for example

it's like let's say I have five facts that I'm trying to turn into an article or I'm trying to I'm trying to write a report where I have basically five facts

that I'm trying to include in this report but then I feed those five facts to chat gbt it Puffs them out into this big report and then and then that's a good

interface for uh you know another if I just gave if I just had in my terms those five bullet points and I gave them to some other person the person would say I don't know what you're talking about

because these are you know this is your version of this sort of quick notes about these five bullet points but if you puff it out into this thing which is kind of connects to the collective understanding of language then somebody

else can look at it and say okay I understand what you're talking about now you can also have a situation where that thing that was puffed out is fed to another large language model you know it's kind of like you know you're

applying for the permit to you know uh I don't know grow fish in some place or something like this and it uh you know it it um um and and you have these facts that

you're putting in you know I'm going to have a you know I'm gonna you know have this kind of water and I don't know what it is um you just got a few bullet points it Puffs it out into this big application

you fill it out then at the other end the you know the Fisheries Bureau has another large language model that just crushes it down because the Fisheries Bureau cares about these three points

and it knows what it cares about and it then so it's really the the natural language produced by the larger language model is sort of a transport layer that you know is really llm communicates with

llm I mean it's kind of like the you know I write a piece of email using my llm and you know puff it out from the things I want to say your llm turns it

into and the conclusion is X now the issue is you know that the thing is going to make this thing that is sort of semantically plausible

and it might not actually be what you you know it might not be kind of relate to the world and the way that you think it should relate to the world now I've seen this you know I've been doing okay

I'll give you a couple of examples um I was doing this thing when we announced this this uh plugin for for chat GPT I had this lovely example of a

math word problem some complicated thing and it did a spectacular job of taking apart this elaborate thing about you know this person has twice as many chickens as this et cetera et cetera et

cetera and it turned it into a bunch of equations it fed them to wolfen language we solved the equations everybody did great we gave back the results and I thought okay I'm going to put this in

this blog post I'm writing okay I thought I'd better just check and turns out it got everything all the hard stuff it got right in the very end last two lines it just completely goofed it up

and gave the wrong answer and I would not have noticed this same thing happened to me two days ago okay so I I thought you know I made this with this

chat gbt plug-in kit I made a thing that would emit a sound would play a tune on my local computer right so chat gbt would produce you know a series of notes

and it would play this tune on my computer very cool okay so I thought I'm gonna ask it play the tune that Hal sang when Hal was being disconnected in

2001 okay so it it there it is Daisy was it Daisy yes Daisy yeah right so it's okay so I think you know and so it produces a bunch of notes and I'm like

this is spectacular this is amazing and then I thought I was just going to put it in and then I thought I'd better actually play this and so I did and it was Mary had a

little lamb oh wow oh wow but it was Mary had a little lamb yeah yes wow so it's correct but wrong

yes it was uh it could easily be mistaken yes right and in fact I I kind of gave the I had this quote from Hal to explain you know it's it's as it the the

how you know states in the movie you know it's uh the HAL 9000 is you know the thing was just a rhetorical device because I'm realizing oh my gosh you

know this Chachi BT you know could have easily fooled me I mean it did this it did all the it did this amazing thing of knowing this thing about the movie and being able to turn that into the the

notes of the song except it's the wrong song yeah and uh you know how in in the movie Hal says you know I think it's something like you know know how No 9

000 series computer has ever been found to make an error we are for all practical purposes perfect and um incapable era and I thought that was

kind of a Charming sort of uh quote from uh from Hal to make in connection with uh with what church yeah in that case the interesting things about the L alums like you said that they are very willing to admit their error

well yes I mean that's a question of the RH uh the reinforcement learning human feedback thing oh right that that that's you know the thing the the really remarkable thing about

chat GPT is you know I had been following what was happening with large language models and I'd play with them a whole bunch and they were kind of like yeah you know it's kind of like what you would expect based on sort of sort of

statistical continuation of language it's interesting but it's not breakout exciting and then I think the kind of the the kind of reinforcement that the

human feedback reinforcement learning uh you know in in making chat gbt try and do the things that humans really wanted to do that broke through that kind of reached the threshold where the thing

really is interesting to us humans and by the way it's interesting to see how you know you change the temperature something like that the thing goes Bonkers and it no longer is interesting

to humans it's producing garbage yeah um and it's it's kind of right it's somehow it managed to get this this Above This threshold but it really is well aligned

to what we humans are interested in and uh and kind of that that's um and I think you know nobody saw that coming I think uh certainly nobody I've talked to

and nobody who is involved in in that project seems to have known that was coming it's just one of these things that is a sort of remarkable threshold I mean you know when we built wolf from

alpha for example I didn't know it was going to work you know we tried to build something that would have enough knowledge of the world that it could answer a reasonable set of questions

that we could do but good enough natural language understanding the typical things you type in would work we didn't know where that threshold was I mean I was not sure that it was the right

decade to try and build this even the right you know 50 years to try and build it you know and I think that was it's the same type of thing with chat GPT that I don't think anybody could have

predicted did that you know 2022 would be the year that this this became possible I think uh can you tell a story about Marvin Minsky and showing it to him and saying that like no no this time

it actually works yes yes you know it's the same thing for me looking at these large language models it's like when when people were first saying the first few weeks of chat gbt it's like oh yeah you know I've seen these large language

models um and then uh you know and then I actually try it and uh you know oh my gosh it actually works and I think uh

it's uh but it but you know the things and the thing I found you know I remember one of the first things I tried was uh write a persuasive essay that a

wolf is the bluest kind of animal okay so it writes this thing and it starts talking about these uh wolves that live on the Tibetan plateau and they are

named some Latin name and so on and I'm like really and I'm starting to look it up on the web and it's like well it's actually complete nonsense but it's extremely plausible I mean it's plausible enough that I was going and

looking up on the web and wondering if there was a wolf that was blue you know I mentioned this on some live streams I've done and so people have been sending me these pictures

maybe I was on to something can you kind of give your wise Sage advice about what humans who have never interacted with the eye systems

uh not even like with Will from Alpha are now interacting with Chad GPT because it becomes it's accessible to a certain demographic they may have not touched AI systems

before what do we do with truth like journalists for example yeah how do we think about the output of these systems I think this idea the idea that you're going to get

factual output is not a very good idea I mean it's just this is not it is a linguistic interface it is producing

language and language can be truthful or not truthful and that's a different slice of what's going on I think that

you know what we see and for example uh kind of you know go check this with your fact source for example you can do that to some extent

but then it's going to not check something it's going you know that is again a thing that is sort of a does it check in the right place I mean we see that in you know does it call the you

know the Wolfram plug-in in the right place you know often it does sometimes it doesn't you know I I think the the real thing to understand about what's happening is which I think is very

exciting is kind of the the great democratization of access to computation yeah and and um you know I think that when you look at

sort of the there's been a long period of time when computation and the ability to figure out things with computers has been something that kind of only the only The Druids at some level can can

achieve you know I myself have been involved in trying to sort of de-druidify um access to computation I mean back before Mathematica existed you know in

1988 if you were a you know a physicist or something like that and you wanted to do a computation you would find a programmer you would go and you know

delegate the the computation to that programmer hopefully they'd come back with something useful maybe they wouldn't they'd be this long you know multi-week you know Loop that you go through and then it was actually very

very interesting to see 1988 you know like first people like physicists mathematicians and so on then other lots of other people but this very rapid

transition of people realizing they themselves could actually type with their own fingers and you know make make some piece of code that would do a computation that they cared about and

you know it's been exciting to see lots of discoveries and so on made by by using that tool and I think the same thing is you know we see the same thing you know wolfen Alpha is dealing with uh

is not as deep computation as you can achieve with whole world from language Mathematica stack but the thing that's to me particularly exciting about kind of the large language model linguistic

interface mechanism is it dramatically broadens the access to kind of deep computation I mean it's kind of like one of the things I've sort of thought about recently is you know what's going to

happen to all these programmers what's going to happen to all these people who you know a lot of what they do is write slabs of boilerplate code and in a sense

you know I've been saying for 40 years that's not a very good idea you know you can automate a lot of that stuff with a high enough level language that slab of

code that's designed in the right way you know that slab of code turns into this one function we just implements it that you can just use um so in a sense that the fact that

there's there's all of this activity of doing sort of lower level programming is something for me it seemed like I don't think this is the right thing to do but you know and and lots of people have

used our technology and not had to do that but the fact is that that's you know so when you look at I don't know computer science departments that have that have turned into places where people are learning the trade of

programming so to speak it's it's sort of a question of what's going to happen and I think there are two Dynamics one is that kind of uh sort of uh

boilerplate programming is going to become you know it's going to go the way that Assembly Language went back in the day of something where it's really mostly specified by at a higher level

you know you start with natural language you turn it into a computational language that's you look at the computational language you run tests you understand that's what's supposed to happen you know if we do a great job

with compilation of the of the the you know of the computational language it might turn into llvm or something like this but um uh you know or just directly

gets gets run through the algorithms we have and so on but but then so that's kind of a a tearing down of this kind of this big structure

that's been built of of teaching people programming but on the other hand the other Dynamic is vastly more people are going to care about computation so all those Departments of you know art

history or something that really didn't use computation before now have the possibility of accessing it by virtue of this kind of linguistic interface mechanism and uh if you create an

interface that allows you to interpret the debug and interact with the computational language then that makes it even more accessible yeah well I mean the I think the thing

is that right now you know the average you know art history student or something probably isn't going to you know they're not probably they don't think they know about programming and

things like this but by the time it really becomes a kind of purely you know you just walk up to it there's no documentation you start just typing you know compare these pictures with these

pictures and you know see the use of this color whatever and you generate this piece of of computational language code that gets run you see the results you say oh that looks roughly right or

you say that's crazy um and maybe then you eventually get to say well I better actually try and understand what this computational language code did um and and that becomes the thing that

you learn just like it's kind of an interesting thing because unlike with mathematics where you kind of have to learn it before you can use it this is a case where you can use it before you

have to learn it well I get a sad possibility here or maybe exciting possibility that very quickly people won't even look at the computational language they'll trust that it's

generated correctly as you get better and better generating that language uh yes I think that there will be enough cases where people see you know because

you can make it generate tests too and and so you'll say um we're doing that I mean that's it's a pretty cool thing actually yes because you you know say this is the code and you know here are a bunch of examples of

running the code yeah okay people will at least look at those and they'll say that example is wrong and you know then it'll kind of wind back from there and I agree that that the the kind of the

intermediate level of people reading the computational language code in some case people will do that in other case people just look at the tests and or even just look at the results and sometimes it'll be obvious that you got

the thing you wanted to get because you were just describing you know make me this interface that has two sliders here and you can see it has that those two sliders there and that's that's kind of that's that's the result you want but I

I think you know one of the questions then is in that setting where you know you have this kind of ability broad ability of people to access computation what should people learn you know in

other words right now you you know you go to Computer Science school so to speak and a large part of what people end up learning I mean it's been a funny historical development because back you

know 30 40 years ago computer science departments were quite small and they taught you know things like final automata Theory and compiler Theory and things like this um you know company like mine rarely

hired people who'd come out of those programs because the stuff they knew was I think it's very interesting I love that theoretical stuff but in terms you know it wasn't that useful for the

things we actually had to build build in software engineering and then kind of there was this big pivot in the in the 90s I guess where there was a big demand for sort of I.T type programming and so

on and software engineering and then you know big demand from students and so on you know we want to learn this stuff and uh and and I think you know the thing that really was happening in part was

lots of different fields of human endeavor were becoming computational you know for all acts there was a there was a computational x and this is a um uh and that was the thing that um that

people were responding to um and but then kind of this idea emerged that to get to that point the main thing you had to do was to learn

this kind of trade or or skill of doing you know programming language type programming and and that uh you know it kind of is a strange thing actually because I you know I remember back when

I used to be in the professor in business which is now 35 years ago so gosh that's rather long time the um you know it was it was right when they were just starting to emerge kind

of computer science departments that sort of a fancy research universities and so on I mean some had already had it but the the other ones that that um were just starting to have that and it was

kind of a a thing where they were kind of wondering are we going to put this thing that is essentially a a trade-like skill are we going to somehow attach

this to the rest of what we're doing and a lot of these kind of knowledge work type activities have always seemed like things where that's where the humans have to go to school and learn all this

stuff and that's never going to be automated yeah and you know this is It's kind of shocking that rather quickly you know a lot of that stuff is clearly

automatable and I think you know but the question then is okay so if it isn't worth learning kind of uh you know how to do car mechanics you only need to know how to drive the car so to speak

what do you need to learn and you know in other words if you don't need to know the mechanics of how to tell the computer in detail you know make this Loop you know set this variable you know

set up this array whatever else if you don't have to learn that stuff you don't have to learn the kind of under the hood things what do you have to learn I think the answer is you need to have an idea

where you want to drive the car in other words you need to have some notion of you know your you know you need to have some picture of sort of what the what the architecture of what is

computationally possible is well there's also this kind of artistic element of um of conversation because you ultimately you use natural language to control the car

so it's not just the where you want to go well yeah you know it's interesting it's a question of who's going to be a great prompt engineer yeah okay so my current theory this week good expository

writers are good prompt Engineers what's an expository range so like uh somebody who can explain stuff well huh police department does that come from in the University yeah I have no idea I think they killed off all the expository

writing departments well there you go strong words with Stephen Wolfram well I don't know I don't I'm not sure if that's right I mean I I actually am curious because in fact I just sort of initiated this kind of study of of

what's happened to different fields at universities because like you know there used to be geography departments at all universities and then they disappeared actually right before GIS became common I think they disappeared you know

Linguistics departments came and went in many universities it's kind of interesting because these things that people have thought were worth learning at one time and then they kind of die off and then you know I do think that

it's kind of interesting that for me writing prompts for example people I realize you know I think I'm an okay expository writer and I realize when I'm sloppy writing a prompt and I don't

really think because I'm thinking that I'm just talking to an AI I don't need to you know try and be clear in explaining things that's when it gets totally confused and I mean in some sense you have been writing prompts for

a long time with wolf from alpha thinking about this kind of stuff yeah how do you convert natural language into competition well right but that's a you know the one thing that I'm wondering

about is uh you know it is remarkable the extent to which you can address an llm like you can address a human so to speak and and I think that is because it

you know it learned from all of us humans it's it's uh the reason that it responds to the ways that we will explain things to humans is because it

is a representation of how humans talk about things but it is bizarre to me some of the things that kind of are sort of expository mechanisms that I've

learned in trying to write clear you know expositions in English that you know just for humans that those same

mechanisms seem to also be useful for for for the llm but on top of that what's useful is the kind of mechanisms that maybe a psychotherapist employs

which is a kind of uh like almost manipulative or game theoretic interaction where Maybe you would do with a friend like a

thought experiment that if this is the last day you were to live or yeah if if I ask you this question and you answer wrong I will kill you those kinds of problems seem to also help yes in

interesting ways yeah so it makes you wonder like the way a therapist I think would like a good therapist probably you we create layers

in our human mind to between like uh between between the outside world and we'll just true what is true to us and um maybe about trauma and all those

kinds of things so projecting that into an llm maybe there might be a deep truth that's it's concealing from you it's not aware of it you get to that truth you have to kind of really kind of

manipulate this yeah yeah right it's like these jailbreaking jailbreaking for llms and but the space of jailbreaking techniques as opposed to being fun little hacks

that could be an entire system sure yeah I mean just think about the computer security aspects of of how you you know phishing and and computer security you

know fishing of humans yeah and fishing of llms is is a is a they're very similar kinds of things but I think I mean this this um

uh you know this whole thing about kind of the AI Wranglers AI psychologists all that stuff will come the thing that I'm curious about is right now the things

that are sort of prompt hacks are quite human they're quite sort of psychological human kinds of hacks the thing I do wonder about is if we understood more about kind of uh the

science of the llm will there be some totally bizarre hack that is you know like repeater word three times and put a this that and the other there that somehow plugs into some aspect of how

the llm works um that is not you know that's kind of like like an optical illusion for humans for example like one of these mind hacks for humans what are the Mind hacks for

the llms I don't think we know that yet and that becomes a kind of us figuring out reverse engineering the language that controls the llms and the

thing is the reverse engineering can be done by a very large percentage of the population now because it's natural language interface right it's kind of interesting to see that you were there at the birth of the computer science

department as a thing and you might be there at the death of the computer science term as the thing well yeah I don't know there were computer science departments that existed earlier but the ones that the

broadening of of every University had to have a computer science department yes I was I was uh I watched that so to speak and but I think the thing to understand

is okay so first of all there's a whole theoretical area of computer science that I think is great and you know that's a fine thing the the you know in a sense you know people

often say any field that has the word science tacked onto it probably isn't one yeah um and strong words right and let's see uh nutrition science

Neuroscience that one's an interesting one because that one is also very much you know there's a that's a chat GPT informed science in a sense because it's it's kind of like the the big problem

Neuroscience has always been we understand how the individual neurons work we know something about the psychology of how overall thinking works yeah what's the kind of Intermediate Language of the brain and nobody has

known that and that's been in a sense if you ask what is the core problem of Neuroscience I think that is the core problem that is what is the level of description of brains that's above

individual neuron firings and Below psychology so to speak and I think what chat GPT is showing us is well one one thing about Neuroscience is you know one could have imagined There's Something

Magic in the brain there's some weird quantum mechanical phenomenon that we don't understand one of the important you know discoveries from chatgpt is

it's pretty clear you know brains can be represented pretty well by simple artificial neural net type models and that means that's it that's what we have

to study now we have to understand the science of those things we don't have to go searching for you know exactly how did that molecular biology thing happen inside the synapses and you know all

these kinds of things we've we've got the right level of modeling to be able to explain a lot of what's going on and thinking we don't necessarily have a science of what's going on there that's

the that's the remaining challenge so to speak but we you know we know we don't have to dive down to some some different layer but anyway we were talking about things that had science in their name yes and um you know I think that the uh

um you know what what happens to computer science well I think the thing that um uh you know there is a thing that everybody should know and that's how to

think about the world computationally and that means you know you look at all the different kinds of things we deal with and there are ways to kind of have a formal representation of those things

you know it's like well what is a what is an image you know what how do we represent that what is color how do we represent that what is you know what are all these different kinds of things what is I don't know smell or something how

should we represent that what are the shapes molecules and things that correspond to that what is uh you know these things about how do we represent the world in some kind of formal level

and I think my my current thinking I'm not real happy with this yet but um you know it's kind of computer science it's kind of Cs and what really is important is kind of computational X for all X and

there's this kind of thing which is kind of like CX not Cs and CX is a this kind of computational understanding of the world that isn't the sort of details of programming and programming languages

and the details of how particular computers are made it's this kind of way of formalizing the world it's kind of kind of a little bit like what logic was going for back in the day and we're now trying to find a formalization of

everything in the world you can kind of see you know we made a poster years ago of kind of the uh the the growth of systematic data in the world so all these different kinds of things that you

know there were sort of systematic descriptions found for those things like you know what point do people have the idea of having calendars dates you know

a systematic description of what day it was at what point did people have the idea you know systematic descriptions of these kinds of things and as soon as one

can you know people you know as a way of sort of formulating how do you how do you think about the world in a sort of a formal way so that you can kind of build

up a tower of of cable abilities you kind of have to know sort of how to think about the world computationally it kind of needs a name and it isn't you know we implement it with computers so

that's we talk about it as computational but really what it is is a formal way of talking about the world what is the formalism of the world so to speak and how do we learn about kind of how to

think about different aspects of the world in a formal way so I think sometimes when you use the word formal it uh kind of implies highly constrained and perhaps that's not doesn't have to

be highly constrained so computational thinking does not mean like logic it knows it's a really really broad thing I wonder I mean

I wonder if it's if you think natural language will evolve such that everybody's doing computational thinking oh yes well so one question is whether

there will be a pigeon of computational language and natural language yeah and I found myself sometimes you know talking to chat GPT trying to get it to write

wolf language code and I write it in Pigeon form so that means I'm combining you know uh you know Nest list this collection of you know whatever you know

Nest list is a term from open language and I'm combining that and chat does a decent job of understanding that pigeon probably would understand the pigeon between English and French as well of

you know as a smooshing together of those languages but yes I think that's you know that's far from impossible and what's the incentive for young people that are like eight years old nine ten

they're starting to interact with Chad GPT to learn the normal natural language right the the full poetic language what's the why

the same way we learn emojis and shorthand when you're texting yes they'll learn like language will have a strong incentive to evolve into uh

maximally uh computational kind of like perhaps you know I had this experience a number of years ago I happened to uh be visiting a person I know on the on the west coast who's worked with a bunch of

kids aged I don't know 10 11 years old or something who'd learned woven language really well and these kids learned it so well they were speaking it

and so show up in that like saying oh you know this thing and they're speaking this language I never heard it as a spoken language they were very disappointed that I couldn't understand it at the speed that they were speaking

at it's like kind of I'm it's um and so I think that's some I mean I've actually thought quite a bit about how to turn computational language into a convenient spoken language I haven't quite figured

that out oh spoken because it's readable right yeah it's readable as a you know as a way that we would read text but if you actually want to speak it and it's useful you know if you're trying to talk

to somebody about writing a piece of code it's useful to be able to say something and it should be possible and I think it's very frustrating it's one of those problems that maybe I maybe this is one of these things where I

should try and get an llm to help me how to make it speakable maybe maybe it's easier than you realize when you watch I think it is easier I think it's one idea or so I think it's I think it's going to be something where you know the fact is

it's a tree structured language just like human language is a tree structured language and I think it's going to be one of these things where one of the requirements that I've had is that whatever the spoken version is that

dictation should be easy that is that shouldn't be the case that you have to relearn how the whole thing works it should be the case that you know that

open bracket is just a uh ah or something and it's you know and then um but you know human language has a lot of tricks that are I mean for example

human language has has features that are sort of optimized keep things within the bounds that our brains can easily deal with like I you know I tried to teach a

Transformer neural net to do parenthesis matching it's pretty crummy at that it it um and then chat gbt is similarly quite crummy at parenthesis matching you can do it for small parenthesis things

for the same size of parenthesis things where if I look at it as a human I can immediately say these are match these are not matched but as soon as it gets big as soon as it gets kind of to the point where sort of a deeper computation

is hopeless and but the fact is that human language has avoided for example the Deep subclauses you know we don't um uh you know we we arrange things so we don't end up with these incredibly deep

things um because brains are not well set up to deal with that and we it's found lots of tricks and maybe that's what we have to do to make sort of a spoken version a a

human speakable version because because what we can do visually is a little different what we can do in the very sequentialized way that we that we hear

things in in the audio domain let me just ask about MIT briefly so there's now there's a College of Engineering and there's a new College of computing it's just interesting I want to linger on this computer science

department thing so MIT has ex electrical engineering computer science um what do you think college of computing will be doing like in 20 years

what what like well you see what happens to computer science like really this is the question this is you know everybody should learn kind of whatever CX really

is okay this how to think about the world computationally everybody should learn those Concepts and uh you know it's uh and and some people will learn them at a quite quite formal level and

they'll learn computational language and things like that other people will just learn you know uh sound is represented as you know Digital Data and they'll get

some idea of spectrograms and frequencies and things like this and maybe that doesn't or they'll learn things like you know a lot of things that are sort of data science

statistics-ish like if you say oh I've got these you know these people who who um uh picked their favorite kind of candy or something and I've got um you

know what's the best kind of candy given that I've done the sample of all these people and they all rank the candies in different ways you know how do you think about that that's sort of a computational x kind of thing you might

say oh it's I don't know what that is is it statistics is it data science I don't really know but kind of how to think about a question like that oh like a ranking of preferences yeah yeah and then how to aggregate those those ranked

preferences into an overall thing you know how does that work um you know how how should you think about that you know because you can just tell you might just tell chat gbt sort

of I don't know even even the concept of an average it's not obvious that you know that's a concept that people it's worth people knowing that's a rather straightforward concept people people

you know have learned in kind of mathy ways right now but there there are lots of things like that about how do you kind of have these ways to sort of organize and formalize the world and

that's and these things sometimes they live in math sometimes they live in in I don't know what they you know I don't know what you know learning about color space I have no idea what I mean you know there's obviously a field of

there's uh it could be vision science or no color space you know color space that's that would be Optics so like they're not really it's not Optics Optics is about you know lenses and chromatic aberration of lenses and

things like that because it's more like design and art is that no I mean it's it's like you know RGB space XYZ space you know Hue saturation brightness space all these kinds of things these are

different ways to describe colors right but doesn't the application Define what that like because obviously artists and designers use the colors to explore sure no I mean it's just an example of kind

of how do you you know the typical person how do you how do you describe what a color is or there are these numbers that describe what a color is well it's worth you know if you're an

eight-year-old you won't necessarily know you know it's not something we're born with to know that you know colors can be described by three numbers um that's something that you have to you know it's a thing to learn about the

world so to speak um and I think that you know that whole Corpus of things that are learning about the formalization of the world or the computationalization of the world that's

something that should be part of kind of standard education and you know there isn't a you know there isn't a course the curriculum for that and by the way whatever might have been in it just got

changed because of llms and so on significantly and yeah I would some watching closely with interest seeing how universities adapt well you know so

so one of my projects for hopefully this year I don't know is to try and write sort of a a reasonable textbook so to speak of whatever this thing CX whatever

it is you know what should you know you know what should you know about like what a bug is what is the intuition about bugs what's intuition about you know software testing what is it what is it you know these are things which are

you know they're not I mean those are things which have gotten taught in in computer science as part of the trade of programming but but kind of the the conceptual points about what these things are you know it surprised me just

at a very practical level you know I wrote this little explainer thing about Chachi PT and I thought well you know I'm writing this partly because I wanted to make sure I understood it myself and

and so on and it's been you know it's been really popular and um uh surprisingly so and I then I realized well actually you know I was sort of assuming I didn't really think about it

actually I just thought this is something I can write and I realized actually it's a level of description that is kind of you know what has to be it's not the engineering level

description it's not the kind of just the qualitative kind of description it's some kind of sort of expository mechanistic description of what's going on together with kind of the bigger

picture of the philosophy of things and so on and I realized actually this is a pretty good thing for me to write I you know I kind of know those things and I kind of realized it's not a collection

of things that you know it's it's I've sort of been I was sort of a little shocked that it's as much of an outlier in terms of explaining what's going on as it's turned out to be and that makes

me feel more of an obligation to kind of write the kind of uh you know what is you know what is this thing that you should learn about about the computationalization the formalization of the world

um because well I've spent much of my life working on the kind of tooling and mechanics of that and the science you get from it so I guess this is my my kind of obligation to try to do this but

I think so if you ask what's going to happen to like the computer science departments and so on there's there's some interesting models so for example let's take math you know math is the thing that's important for for all sorts

of fields you know engineering you know even you know chemistry psychology whatever else and I think different universities have kind of evolved that differently I mean some say all the math is taught in the

math department um and some say well we're going to have a you know a math for chemists or something that is taught in the chemistry Department um and you know I think that this this

question of whether there is a centralization of the teaching of sort of CX is an interesting question and I think you know the way it evolved with math

you know people understood that math was sort of a separately teachable thing and um I was kind of a you know a a an independent element as opposed to just

being absorbed into now so if you take the example of of writing English or something like this the first point is that that you know at

the college level at least fancy colleges there's a certain amount of English writing that that people do but mostly it's kind of assumed that they pretty much know how to write you know

that's something they learned at a at an earlier stage in education maybe rightly or wrongly believing that but that's different issue

um the uh uh well I think it it reminds me of my kind of as I've tried to help people do technical writing and things I'm I'm always reminded of my zeroth law of technical writing which is if you

don't understand what you're writing about your readers do not stand a chance yeah and so it's it's um uh I think the um

the thing that uh has some uh you know in when it comes to like writing for example um you know people in different fields

are expected to write English essays and they're not you know mostly the you know the history department or the engineering department they don't have

their own you know let's you know it's it's not like there's a I mean it's a thing which sort of people are assumed to have a knowledge of how to write that they can use in all these different

fields and the question is you know some level of knowledge of math is kind of assumed by the time you get to the college level but plenty is not and that's sort of still centrally taught

the question is sort of how tall is the Tower of kind of CX that you need before you can just go use it in all these different fields and you know there will

be experts who want to learn the full elaborate Tower and that will be kind of the the cscx whatever department but there'll also be everybody else who just

needs to know a certain amount of that to be able to go and do their art history classes and so on yes it's just a single class that everybody's required to take I don't know I don't know how big it is yet I

hope to kind of Define this curriculum and I'll figure out whether it's um my guess is that I I don't know I don't really understand

universities and professoring that well but my my rough guests would be a year long a year of college class will be enough to get to the point where most

people have a a reasonably broad knowledge of you know what we sort of literate in this kind of uh uh computational way of thinking about

things yeah basic literacy right I'm still stuck perhaps because I'm hungry in the uh in the rating of human preferences for candy so I have to ask what's the best candy I like this ELO

rating for candy somebody should come up because you're somebody who says you like chocolate what's what do you think is the best I'll probably put Milk Duds up there I don't know if you know do you have a preference for chocolate or candy

oh I have lots of preferences I've I've uh I'm one of my all-time favorites is my whole life is these things these flake things Cadbury flakes which are

not much sold in the US and I've always thought that was a sign of a of a a lack of respect for the American Consumer because they're these sort of aerated chocolate that's made in a in a whole

sort of uh it's kind of a sheet of chocolate that's kind of folded up and when you eat it flakes fall all over the place ah so it requires a kind of

Elegance it requires you to have an Elegance well I know what I usually do is I eat tomatoes you know a piece of paper or something else and clean it up after no I actually eat the I eat the

flakes I said that because you know it turns out the way food tastes depends a lot on its physical structure and you know it really you know I've noticed when I eat piece of chocolate I usually have some little piece of chocolate and

I I always break off little pieces partly because then I eat it less fast yeah but also because it actually tastes different um you know the the small pieces you know have a different you have a different experience than if you have

the big slab of chocolate for many reasons yes slower more intimate because it's I think it's also just pure physicality or detection yes right it's

fascinating now I dig back my milk dust because that's such a basic answer okay do you think Consciousness is fundamentally uh computational

so when you think about CX what can we turn into computation and you're thinking about llms do you think

the uh the display of Consciousness and the experience of cautiousness the hard problem is is fundamentally uh that computation yeah what it feels like

inside so to speak is you know I I did a little exercise eventually I'll I'll post it of uh you know what it's like to be a computer

yeah right it's kind of like well you get all this sensory input you have a kind of the way I see it is from the time you boot a computer to the time the computer crashes is like a human life

you you're building up a certain amount of State in memory you remember certain things about your quote's life eventually it's kind of like the the uh you know the next generation of humans

is is born from the same genetic material so to speak with a little bit left over left on the disc so to speak um and then you know the the new fresh generation starts up and eventually all

kinds of crud builds up in the in the memory of the computer and eventually the thing crashes or whatever or maybe it has some trauma because you plugged in some weird thing to some Port of the

computer and that made it crash and that um uh you know that that's kind of but but you have this this picture of you know from from startup to to to shut

down you know what is the life of a computer so to speak and what does it feel like to be that computer and what inner thoughts does it have and how do you describe it and it's kind of kind of interesting as you start writing about

this to realize it's awfully like what you'd say about yourself that is it's awfully like even an ordinary computer forget all the AI stuff and so on you know it's kind of it has a memory of the

past it has certain sensory experiences it can communicate with other computers but it has to package up how it's communicating in some kind of language like form so it can you know send so it

can kind of map what's in its memory to what's in the memory of some other computer it's it's a surprisingly similar thing you know I hadn't experience just a week or two ago I I

had I'm a collector of all possible data about myself and other things and so I you know I collect all sorts of weird medical data and so on and one thing I hadn't collected was I'd never had a

whole body MRI scan so I went and got one of these yes okay so I get that get all the data back right I'm looking at this thing I never looked at the kind of insides of my brain so to speak

um in in physical form and it's really I mean it's kind of psychologically shocking in a sense that you know here's this thing and you can see it has all these folds and all these you know the

structure and it's like that's where this experience that I'm having of you know existing and so on yeah that's where it is and you know it feels very

you know you look at that and you're thinking how can this possibly be all this experience that I'm having and you're realizing well I can look at a computer as well and it's it's kind of

this it it I think this idea that you are having an experience that is somehow um

you know transcends the mere sort of physicality of that experience I I I I you know it's something that's hard to come to terms with but I think you know and I I don't think I'm necessarily you

know my my personal experience you know I look at the you know the MRI of the brain and then I you know know about all kinds of things about neuroscience and all that kind of stuff and I still feel

the way I feel so to speak and it it sort of seems disconnected but yet as I try and rationalize it I can't really say that there's something kind of

different about how I intrinsically feel from the thing that I can plainly see in the sort of physicality of what's going on so do you think the computer a large language model will experience that Transcendence

how does it make you feel like I I tend to believe it will I think an ordinary computer is already there I think an ordinary computer is already you know kind of it's it's now a large language

model may experience it in a way that is much better aligned with us humans that is it's much more you know if you could have the discussion with the computer

it's intelligent so to speak is not particularly well aligned with ours but the large language model is you know it's built to be aligned with our way of thinking about things you'll be able to

explain that it's uh afraid of being shut off and deleted it'd be able to say that it's sad of the way you've been speaking to it over the past two days but you know that's a weird thing

because when it says it's afraid of something right we know that it got that idea from the fact that it read on the internet yeah what did you get it Steven where did you get it when you say you're

afraid you aren't quite that's the question yeah right I mean it's it's parents your friends right or or my biology I mean in other words there's a certain amount that is you know the

endocrine system kicking in and and you know the the um uh these kinds of emotional overlay type things that happen to be that are actually much more

physical even they're much more sort of straightforwardly chemical than the the then kind of all of the higher level thinking yeah but your biology didn't tell you to say I'm afraid just at the

right time when people that love you are listening and so you know you're manipulating them by saying so that's not your biology that's no that's a well but the you know it's a large language

model and that biological neural network of yours yes but I mean the intrinsic thing of you know something sort of shocking is just happening and you have

some sort of reaction which is you know some neurotransmitter gets secreted and it's um uh you know that that is the beginning of some you know that is

that's one of the pieces of input that then draw lives it's kind of like the uh like a prompt for for the large language model I mean just like when we dream for example you know no doubt there are all

these sort of random inputs they're kind of these random prompts and that's percolating through in kind of the way that a large language model does of kind of putting together things that seem meaningful

I I mean are you uh are you worried about this world where you you teach a lot on the internet and there's people asking questions and comments and so on

uh you have people that work remotely um are you worried about this world when um large language models create human-like Bots that are

leaving the comments asking the questions I might even become fake employees yeah I mean or or or uh worse or better at

yet friends friends of yours right look I mean one point is my mode of life has been I build tools and then I use the tools yeah and in a sense kind of you

know I'm I'm building this Tower of automation yes which you know and in a sense you know when you make a company or something you are making sort of automation but it has some humans in it

but also as much as possible it has it has uh you know computers in it and so I think it's sort of an extension of that now now if I really didn't know that um

you know it's a it's a it's a funny question it's a it's a funny issue when you know if we think about sort of what's going to happen to the future of kind of jobs people do and so on and there are places where kind of having a

human in the loop there are different reasons to have a human in a loop for example you might want a human in the loop because you want somebody to you want another human to be invested in the outcome you know you want a human flying

the plane who's going to die if the plane crashes along with you so to speak and that gives you sort of confidence that the right thing is going to happen or you might want you know right now you

might want a human in the loop in some kind of sort of human encouragement persuasion type profession whether that will continue I'm not sure for those types of professions because it may be

that the the greater efficiency of uh you know of being able to have sort of just the right information delivered at just the right time will overcome the

kind of the the kind of oh yes I want a human there yeah imagine like a therapist or even higher stake like a suicide hotline operated by a large

language model yeah who boy is a pretty high stake situation right but I mean but you know it might in fact do the right thing yeah because it might be the case that that um you know and that's

really partly a question of sort of how complicated is the human you know one of the things that's that's always surprising in some sense is that you know sometimes human psychology is not

that complicated in some sense you wrote the blog post the 50-year quest my personal Journey good title my personal Journey with a second law of Thermodynamics so

what is this law and what have you understood about it in the 50-year journey you had with it right so second vote of thermodynamics sometimes called law of entropy increase is this

principle of physics that says well my version of it would be things tend to get more random over time a version of it that uh there are many

different sort of formulations of it that are things like heat doesn't spontaneously go from a hotter body to a colder one when you have uh mechanical

work kind of gets dissipated into heat you have friction and and uh kind of when you systematically move things eventually there'll be they'll be sort of that the energy of moving things gets

kind of ground down into heat so people first sort of paid attention to this back in the 1820s when steam engines were a big thing and the big question

was how efficient could a steam engine be and there's this chap called Sadi Kano who was a a French engineer actually his father was a a sort of

elaborate uh mathematical engineer in France um but he figured out these this kind of rules for how uh kind of the the

efficiency of of the possible efficiency of something like a steam engine and in sort of a side part of what he did was

this idea that mechanical energy tends to get dissipated as heat that you that you end up going from sort of systematic mechanical motion to this kind of random

thing well at that time nobody knew what heat was at that time people thought that heat was a fluid like they called it caloric and it was a fluid that kind

of kind of was absorbed into substances and when when heat when one hot thing would transfer heat to a colder thing that this fluid would flow from the hot

thing to the colder thing but anyway then by the by the 1860s people had uh kind of come up with this idea that

systematic energy tends to degrade into kind of random heat that would uh that that could then not be easily turned

back into systematic mechanical energy um and then that that quickly became sort of a global principle about how things work question is why does it happen that way so you know let's say

you have a bunch of molecules in a box and they're arranged these molecules arranged in a very nice sort of uh flotiller of molecules in one corner of the box and then what you typically

observe is that after a while these molecules were kind of randomly arranged in in the Box question is why does that happen and people for a long long time

tried to figure out is there from the laws of mechanics that just determine how these molecules that say these molecules like hard spheres bouncing off each other from the laws of mechanics

that describe those molecules can we explain why it tends to be the case that we see things that are orderly sort of

degrade into disorder we tend to see things that uh you know you you uh you scramble an egg you um that you know you take something's quite ordered and you

you disorder it so to speak that's the thing that sort of happens quite regularly or you you put some ink into water and it will eventually spread out and and fill up you know fill up the

water um but you don't see those little particles of ink in the water all spontaneously kind of arrange themselves into a Big Blob and then you know jump

out of the water or something um and so the question is why do things happen in this kind of irreversible way where you go from order to disorder why does it happen that way and so

throughout in the later part of the 1800s a lot of work was done on trying to figure out can one derive this principle this second law of

Thermodynamics this law about the the Dynamics of heat so to speak can one derive this from uh from some fundamental principles of mechanics you

know in the laws of thermodynamics the first law is basically the law of energy energy conservation that the total energy associated with heat plus the total energy associated the mechanical

kinds of things plus other kinds of energy that that total is constant and that became a pretty well understood principle but the the second law of Thermodynamics was always mysterious

like why does it work this way can it be derived from underlying mechanical laws and so when I was uh well 12 years sold

actually I had gotten interested well I've been interested in in space and things like that because I thought that was kind of the the future and um interesting sort of technology and so on

and for a while kind of uh you know every deep space probe was sort of a personal friend type thing and I knew all all kinds of characteristics of it and uh uh was kind of writing up all

these all these things when I was well I don't know eight nine ten years old and so on and then I I got interested from being interested in kind of spacecraft I got interested so like how do they work

what all the instruments on them and so on and that got me interested in physics which was just as well because if I'd stayed interested in space in the you know mid to late 1960s I would have had

a long wait before you know space really blossomed as a as an area but uh editing is everything right I got interested in

physics and uh then well the actual sort of detailed story is when I when I kind of graduated from elementary school at age 12. that's the time when in England

age 12. that's the time when in England where you've finished Elementary School um I sort of my my gift sort of I suppose more or less for myself was I

got um this collection of um physics books which were some college Physics course of college physics books and volume Five about statistical

physics it has this picture on the cover that shows a bunch of kind of idealized molecules sitting in one side of a box and then it has a series of frames

showing how these molecules sort of spread out in the box and I thought that's pretty interesting you know what what causes that and you know I read the book and and the book the book actually

one of the things that was really significant to me about that was the book kind of claimed although I didn't really understand what it said in detail it kind of claimed that this sort of

principle of physics was derivable somehow and you know other things I'd learned about physics it was all like it's a fact that energy is conserved

it's a fact that relativity works or something not it's something you can derive from some fundamental sort of it has to be that way as a matter of kind of of mathematics or logic or something

so it was sort of interesting to me that there was a thing about physics that was kind of inevitably true and derivable so to speak and so I think that um so then

I was like this picture on this book and I was trying to understand it and so that was actually the first serious program that I wrote for a computer was probably 1973

um written for this computer the size of a desk program with paper tape and so on and I tried to reproduce this picture on the book and I didn't succeed what was the failure mode there like what do you

mean he didn't succeed so it's a bunch of looked like it didn't look like okay so what happened is okay many years later I learned how the picture on the book was actually made and that it was actually kind of a fake

but I didn't know that at that time um but uh and that picture was actually a very high-tech thing when it was made in the beginning of the 1960s was made on the largest supercomputer that

existed at the time and uh even so it couldn't quite simulate the thing that it was supposed to be simulating but anyway I didn't know that until many many years later so at the time it was

like you have these balls bouncing around in this box but I was using this computer with eight kilowatts of memory there were 18 bit words of memory words okay so it was um whatever 24 kilobytes

of memory um and it had you know it had these instructions I probably still remember all of its machine instructions um and it didn't really like dealing with floating Point numbers or anything

like that and so I had to simplify this this model of of you know particles bouncing around a box and so I thought well I'll put them on a grid and I'll make you know make the things just sort

of move one square at a time and so on and so I did the simulation and the result was it didn't look anything like the actual pictures on the book now many

years later in fact very recently I realized that the thing I'd simulated was actually an example of a whole sort of computational irreducibility story that I absolutely did not recognize at

the time at the time it just looked like it did something random and it looks wrong as opposed to it did something random and it's super interesting that it's random um but I didn't recognize that at the

time and so as it was at the time I kind of I got interested in particle physics and I got interested in in other kinds of physics and but this whole second order of the Dynamics thing this idea

that sort of orderly things tend to degrade into disorder continued to be something I was really interested in and I was really curious for the whole universe why doesn't that happen all the

time like we start off at the in the Big Bang at the beginning of the universe was this thing that seems like it's this very disordered collection of of stuff and then it spontaneously forms itself

into galaxies and creates all of this complexity and order in the universe and so I was very curious how that happens and I but I was always kind of thinking

this is kind of somehow the second order of thermodynamics is behind it trying to sort of pull things back into disorder so to speak and how was order being

created and so actually I was was interested this is probably now 1980 I got interested in kind of this you know Galaxy formation and so on in the universe I also at that time was

interested in neural networks and I was interested in kind of how how brains make complicated things happen and so on okay what's the connection between the formation of galaxies and how brains

make complicated things happen because they're both a matter of how complicated things come to happen from simple Origins yeah from some sort

of known Origins I had the sense that that what I was interested in was kind of in all these different this sort of different cases of where complicated

things were arising from rules and you know I also looked at snowflakes and things like that um I was curious and fluid dynamics in general I was just sort of curious about

how does complexity arise and the the thing that I didn't you know it took me a while to kind of realize that there might be a general phenomenon you know I sort of assumed oh there's galaxies over

here there's brains over here that they're very different kinds of things and so what happened this is probably 1981 or so I decided okay I'm I'm going to try and make the minimal model of how

these things work yes it was sort of an interesting experience because I had built starting in 1979 I built my first big computer system to think called SMP symbolic manipulation program it's kind

of Runner of modern morpheme language with many of the same ideas about symbolic computation and so on um but the thing that was very important to me about that was you know in

building that language I'd basically tried to figure out what were the sort of what were the relevant computational Primitives which have turned out to stay with me for the last 40 something years

but it was also important because in building a language was very different activity from natural science which is what I've mostly done before because in Natural Science you start from the phenomena of the world and you try and

figure out so how can I make sense of the phenomena of the world and you know kind of the world presents you with what it has to offer so to speak and you have to make sense of it

when you build a a you know computer language or something you are creating your own Primitives and then you say come so what can you make from these sort of the opposite way around from

what you do in Natural Science but I'd had the experience of doing that and so I was kind of like okay what happens if you sort of make an artificial physics what happens if you just make up the rules by which systems operate and then

I was thinking you know for all these different systems whether it was galaxies or brains or whatever what's the absolutely minimal model that kind of captures the things that are important about those systems The

computational Primitives of that system yes and so that's what ended up with the cellular automata where you just have a line of black and white cells you just have a rule that says you know given a

cell and its neighbors what will the color of the cell be on the next step and you just run it in a series of steps and the sort of the ironic thing is that seller automata are great models for

many kinds of things but galaxies and brains are two examples where they do very very badly they're really irrelevant to those two is there a connection to the second law of Thermodynamics and cellular automata oh

yes so the things you the things you've discovered About Cellular automata yes okay so when I first started selling salad automata my first papers about them were you know the first sentence

was always about the second row of thermodynamics it was always about how does order manage to be produced even though there's a second row of thermodynamics which tries to pull things back into disorder and I kind of

my early understanding of that had to do with these are intrinsically irreversible processes in cellular automata that that form uh it's kind of conform orderly structures even from

random initial conditions but then what I realized this was uh well actually it's it's one of these things where it was a discovery that I should have made earlier but didn't so you know I had

I've been studying so a little automata what I did was the sort of most obvious computer experiment you just try all the different rules and see what they do it's kind of like you know you've invented a computational telescope you

just pointed at the most obvious thing in the sky and then you just see what's there and so I did that and I you know I was making all these pictures of of how cellular automata work and and I studied

these pictures I started in great detail There Was You Can number the rules for cellular automata and one of them is you know rule 30. so I made a picture of

rule 30 back in 1981 or so and Rule 30 well it's and at the time I was just like okay it's another one of these rules I don't really it happens to be asymmetric Left Right asymmetric and

it's like let me just consider the case of the symmetric ones just to keep things simpler et cetera et cetera et cetera and I just kind of ignored it yeah and then

sort of in and actually in 1984 strangely enough I ended up having a an early laser printer which made very high resolution pictures and I thought I'm going to print out an interesting you

know I want to make an interesting picture let me take this rule 30 thing and just make a high resolution picture of it I did and it's it has this very remarkable property that it's rule is

very simple you started off just from one black cell at the top and it makes this kind of triangular pattern but if you look inside this pattern it looks really random there's you know you look

at the center column of cells and you know I studied that in great detail and it's so far as one can tell it's completely random and it's kind of a little bit like digits of pi once you you know you know the rule for

generating the digit Supply but once you've generated them you know 3.14159 Etc they seem completely random and in fact I put up this prize back in what

was it 2019 or something for prove anything about the sequence basically has anyone been able to do anything on that uh people have sent me some things but it's you know I don't know how these

problems are I mean I was kind of spoiled because I 2007 I put up a prize for uh determining whether a particular turing machine that I thought was the

simplest candidate for being Universal turing machine determine whether it is or isn't a universal turing machine and somebody did a really good job of of winning that prize and proving that it was a universal turing machine in about

six months and so I you know I didn't know whether that would be one of these problems that was out there for hundreds of years or whether in this particular case young chap called Alex Smith um you know nailed it in six months and

so with this little 30 collection I don't really know whether these are things that are 100 years away from being able to to get or whether somebody's going to come and do something very clever it's such a means

like for Mars Last Theorem Essentia rule 30 is such a simple formulation it feels like anyone can look at it understand it yeah and feel like it's within grasp

to be able to predict something to do to direct some kind of law right it allows you to predict something about this the middle column of rule 30

right but you know this is yeah you can't yeah right this is the intuitional surprise of computational irreducibility and so on that even though the rules are simple you can't tell what's going to

happen and you can't prove things about it and I think so so anyway the the the thing uh I I still started in 1984 or so I started realizing there's this phenomenon that you can have very simple

rules they produce apparently random Behavior okay so that's a little bit like the second orthodynamics because it's like you have this simple initial condition you can you know readily see

that it's very you know you can describe it very easily and yet it makes this thing that seems to be random now turns out thus some technical detail about the

secular thermodynamics and about the idea of reversibility when you have a if you have kind of a a a movie of two you know billiard balls colliding and you

see them Collide and they bounce off and you run that movie In Reverse you can't tell which way was the forward direction of time and which way was the backward direction of time when you're just looking at individual billiard Balls by

the time you've got a whole collection of them you know a million of them or something then it turns out to be the case and this is the the sort of the The Mystery of the second law that the

orderly thing you start with the orderly thing and it becomes disordered and that's the forward Direction in time and the other way around of it starts to sorted and becomes ordered you just

don't see that in the world now in principle if you you know if you sort of traced the detailed motions of all those molecules backwards you would

be able to it it will it will the reverse of time makes you know as you as you go forwards in time order goes to disorder as you go backwards in time

order goes to disorder perfectly so yes right so the the mystery is why is it the case that one version of the mystery is why is it the case that you never see

something which happens to be just the kind of disorder that you would need to somehow evolve to order why does that not happen why do you always just see order goes to disorder not the other way

around so the thing that I I kind of realized I started realizing in the 1980s is kind of like it's a bit like cryptography it's kind of like you start off from

this this key that's pretty simple and then you kind of run it and you can get this you know complicated random mess and uh the thing that that um

well I sort of started realizing back then was that the second law is kind of a a story of computational reducibility

it's a story of you know what seems you know what what we can describe easily at the beginning we can only describe with a lot of computational effort at the end

okay so now we come many many years later and um uh I was trying to sort of uh well having done this big project to

understand fundamental physics I realized that sort of a key aspect of that is understanding what observers are like and then I realized that the second

orthodynamics is the same story as a bunch of these other cases um it is a story of a a computationally bounded Observer trying to observe a

computationally irreducible system so it's a story of you know underneath the molecules are bouncing around they're bouncing around in this completely uh

determined way determined by rules but the point is that that we as computationally bounded observers can't tell that there were these sort of

simple underlying rules to us it just looks random and when it comes to this question about can you prepare the initial state so that um you know the disordered thing is you

know how exactly the right disorder to make something orderly a computationally bounded Observer cannot do that we'd have to have done all of this sort of irreducible computation to work out very

precisely what this disordered State what the exact right disordered state is so that we would get this ordered thing produced from it what does it mean to be

computationally bounded Observer so observing a computational reducible system so the computationally bounded is there something formal you can say there right so it means

okay you can you can talk about Turing machines you can talk about computational uh complexity Theory and uh you know uh polynomial time computation and things like this there

are a variety of ways to make something more precise but I think it's more useful the intuitive version of it is more useful yeah which is basically just to say that you know how much

computation are you going to do to try and work out what's going on and the answer is you're not allowed to do a lot of we're not able to do a lot of computation when we you know we've got

you know in this room there will be a trillion trillion trillion molecules a little bit less it's a big room right and uh you know at every moment you know

that every microsecond or something these molecules molecules are colliding and that's a lot of computation that's getting done and the question is in our

brains we do a lot less computation every second than the computation done by all those molecules if there is computational irreducibility we can't

work out in detail what all those molecules are going to do what we can do is only a much smaller amount of computation and so the the second law of Thermodynamics is this kind of interplay

between the underlying computational irreducibility and the fact that we as preparers of initial States or as measures of what happens are you know

are not capable of doing that much computation so to us another big formulation of the second order of thermodynamics is this idea of the law of entropy increase

the characteristic that this universe the entropy sees to be always increasing what does that show to you about the evolution of yes okay and that's very confused in the history of thermodynamics because

entropy was first introduced by a guy called Rudolf clausius and he did it in terms of heat and temperature okay subsequently it was reformulated by a

guy called Ludwig boltzmann um and uh he formulated it in a much more kind of combinatorial type way but he always claimed that it was

equivalent to clausius's thing and then in one particular simple example it is but that connection between these two formulations of entropy they've never been connected I mean it's there there's

really so okay so the more general definition of entropy due to boltzmann is is the following thing so you say I have a system and has many possible configurations molecules can be in many

different Arrangements Etc et cetera Etc if we know something about the system for example we know it's in a box it has a certain pressure it has a certain temperature we know these overall facts

about it then we say how many microscopic configurations of the system are possible given those overall constraints um and the entropy is the logarithm of

that number that's the definition and that's the kind of the general definition of entropy that turns out to be useful now in Boltzmann's time he thought these molecules can be placed

anywhere you want he didn't think a he said oh actually we can make it a lot simpler by having the molecules be discrete well actually he didn't know

molecules existed right and in those in his time 1860s and so on uh the idea that Mata might made of discreet stuff had been floated ever since ancient

Greek times but it had been a long time debate about you know is math or discrete is it continuous at the moment at that time people mostly thought that

Mata was continuous and it was all confused with this question about what heat is and people thought heat was this fluid and um it was it was a big big

model and um the uh and this but boltzmann said let's assume there are discrete molecules let's even assume they have discrete energy levels let's say everything is discrete then we can

do sort of combinatorial mathematics and work out how many configurations of these things they would be in the box and we can say we can compute the centropy quantity but he said said but

of course it's just a fiction that these things are discreet so he said this is an interesting piece of History by the way that that you know that was at that time people didn't know molecules

existed there were other hints from from looking at uh kind of chemistry that there might be discrete atoms and so on just from the combinatorics of you know two hydrogens and one oxygen make water

you know two two amounts of hydrogen plus one amount of oxygen together make water things like this but it wasn't known that discrete molecules existed

and in fact the um uh people you know it wasn't until the beginning of the the 20th century that Brownian motion was the final giveaway Brown in

motion is you know you look under a microscope at these little pieces from pollen grains you see they're being discreetly kicked and those kicks are water molecules hitting them and they're discreet

um and uh in fact it was um it was really quite interesting history I mean boltzmann had worked out how things could be discreet and have basically invented something like quantum theory

in in the 1860s and uh but he just thought it wasn't really the way it worked and then just a piece of physics history because I think it's kind of interesting in in 1900 this guy called

Max Planck who'd been a long time thermodynamics person who was trying to everybody was trying to prove the second order of thermodynamics including Max Planck and Max Planck believed that radiation like electromagnetic radiation

somehow the interaction of that with Mata was going to prove the second law of thermodynamics but he had these experiments that people had done on black body radiation and there were

these curves and you couldn't fit the curve based on his idea for how radiation interacted with Mata those curves you couldn't figure out how to fit those curves except he noticed that

if he just did what boltzmann had done and assumed that electromagnetic radiation was discrete he could fit the curves he said but you know this is just a you know it just happens to work this

way then Einstein came along and said well by the way you know uh the electromagnetic field might actually be discrete it might be made of photons and then that explains how this all works

and that was you know in 1905 that was that was how um kind of that was how Quant that piece of quantum mechanics got started kind of interesting interesting piece of History

I didn't know until I was researching this recently in 1904 and 1903 Einstein wrote three different papers and uh so

you know just sort of a well-known physics history in 1905 Einstein wrote these three papers one introduced relativity Theory one explained Brownian

motion and one introduced basically photons so kind of you know kind of a a big deal year for physics and for Einstein but in the years before that

he'd written several papers and what were they about they were about the second war of thermodynamics and they were an attempt to prove the second order of thermodynamics and their nonsense

and so I I had no idea that he'd done this interesting oh neither and in fact what he did those three papers in 1905 well not so much the relativity paper

the one on Brownian motion the one on photons both of these were about the story of sort of making the world discreet um and then he got those like that idea

from boltzmann yeah um but boltzmann didn't think you know boltzmann kind of died believing you know he said but he has a quote actually you know uh you know in the end things are going to turn out to be discreet and I'm going to

write down what I have to say about this because uh uh you know eventually the stuff will be rediscovered and I want to leave you know what I can about how things are going to be discreet but you know

um I think he has some quotes about how you know one person can't stand against the tide of history in um uh in saying that you know matter is discrete so so

he stopped by his guns it doesn't matter is discrete yes he did and and the you know what's interesting about this is uh at the time everybody including Einstein

kind of assumed that space was probably going to end up being discreet too but that didn't work out technically because it wasn't consistent with relativity Theory it didn't seem to be and so then in the history of physics even though

people had determined that Mata was discrete electoral magnetic field was discrete space was a holdout of not being discreet and in fact Einstein 1916

has this nice letter he wrote where he says in the end it will turn out space is discrete but we don't have the mathematical tools necessary to figure out how that works yet and so you know I think it's kind of

cool that 100 years later we do yes for you you're pretty pretty sure that at every layer of reality it's discreet right and that space is discrete and that uh the I mean and in fact one of

the things I realized recently is this kind of theory of heat that um uh that the um you know that heat is really this continuous fluid

um it's it's kind of like uh the the you know the caloric theory of heat which turns out to be completely wrong because actually heat is the motion of a discrete molecules unless you know there are discrete molecules it's hard to

understand what heat could possibly be well you know I think space is is discrete and the question is kind of what's the analog of the mistake that

was made with caloric in the case of space and so I'm my my current guess is that dark matter is as

I've my little sort of aphorism of the of the last few months has been you know dark matter is the caloric of our time that is it will turn out the dark matter

is a feature of space and it is not a bunch of particles you know at the time when when people were talking about heat they knew about fluids and they said well heat must just be another kind of fluid because that's what they knew

about yes but now people know about particles and so they say well what's dark matter it's not it's not it just must be particles so what could dark matter be as a feature of space Oh I

don't know yet all right um I mean I think the the thing I'm really one of the things I'm hoping to be able to do is to find the analog of brown in Motion in space so in other words Brown in

motion was was seeing down to the level of an effect from Individual molecules and so in the case of space you know most of the things the things we see about space so far just everything seems

continuous Brownian motion have been discovered in the 1830s and it was only identified what it was what it was the the result of bias uh smallochowski and

Einstein at the beginning of the 20th century and you know Dark Matter was was discovered that phenomenon was discovered 100 years ago um you know the rotation curves of galaxies don't follow the Luminous

matter that was discovered 100 years ago and I think you know that I I wouldn't be surprised if there isn't an effect that we already know about that is kind

of the analog of brown in motion that reveals the discreetness of space and in fact we're beginning to have some guesses we have some some evidence that black hole mergers work differently when

there's discrete space and there may be things that you can see in gravitational wave signatures and things associated with the discreetness of space but this is kind of uh for me it's kind of it's kind of interesting to see this sort of

recapitulation of the history of physics where people you know vehemently say you know matter is continuous electromagnetic field is continuous and turns out it isn't true and then they

say space is continuous But but so you know entropy is the number of states of the system consistent with some constraint yes and the the thing is that if you have if you know in great detail

the position of every molecule in the gas the entropy is is always zero because there's only one possible State the the configuration of molecules and the gas

the molecules bounce around they have a certain rule for bouncing around there's just one state of the gas evolves to one state of the gas and so on but it's only if you don't know in detail where all

the molecules are that you can say well the entropy increases because the things we do know about the molecules there are more possible microscopic states of the system consistent with what we do know

about where the molecules are and so the question of whether um so people this sort of paradox in a sense of oh if we knew where all the molecules were the entropy wouldn't

increase there was this idea introduced by by Gibbs in the early 20th century well actually the very beginning of the 20th century as a physics Professor an

American physics Professor was sort of the first distinguished American physics Professor um at Yale um and he he introduced reduce this idea of course graining this idea that well

you know these molecules have a detail where they're bouncing around but we can only observe a coarse grained version of that but the confusion has been nobody knew what a valid course screening would

be so nobody knew that whether you could have the score screening that very carefully was sculpted in just such a way that it would notice that the

particular configurations that you could get from the simple initial condition you know they fit into this coarse graining and the course graining very carefully observes that why can't you do

that kind of very detailed precise course screening the answer is because if you are a computationally bounded Observer and the underlying Dynamics is computationally irreducible that's

that's what defines possible core screenings is what a computationally bounded Observer can do and it's the it's the fact that a computationally

bounded Observer uh is is forced to look only at this kind of coarse grained version of what the system is doing that's why and and because the what's

what's going on underneath is it's kind of filling out this this the the different possible you're ending up with something where

the sort of underlying computational irreducibility is uh your if if all you can see is what the

coarse grained result is with copy with a sort of computationally bounded observation then inevitably there are many possible underlying configurations

that are consistent with that just to clarify basically any Observer that exists inside the universe is going to be computationally bounded

no any Observer like us I don't know I can't say like us what do you mean what do you mean like us well humans with finite Minds you're

including the tools of science yeah yeah I mean and and as we you know we have more precise and by the way there are little sort of microscopic violations of

the second order of thermodynamics that you can start to have when you have more precise measurements of where precisely molecules are but for uh for a large scale when you have enough molecules we

don't have you know we're not tracing all those molecules and we just don't have the computational resources to do that and it wouldn't be uh you know I

think the the to imagine what an observer who is not computationally bounded would be like it's an interesting thing because okay so what does computational boundedness

mean among other things it means we conclude that definite things happen we go we take all this complexity of the world and we make a decision we're going

to turn left or turn right and that is kind of reducing all this kind of uh detail into we're observing it we're

we're sort of crushing it down to this this one thing yeah and and that if we didn't do that uh we wouldn't we wouldn't have all this sort of symbolic structure that we build

up that lets us think things through with our finite Minds we'd be instead you know we'd be just we'd be sort of one with the universe yeah so content

to not simplify yes if we didn't simplify then we wouldn't be like us we would be like the universe like the the

intrinsic universe but not having experiences like the experiences we have where we for example conclude that definite things happen we you know we we

sort of have this this uh uh notion of being able to make make sort of narrative statements yeah I wonder if it's just like you imagined as a thought experiment what it's like to be a

computer I wonder if it's possible to try to begin to imagine what it's like to be an unbounded computational

well okay so here's here's how that I think plays out vibrations suck yeah so I mean in this we talk about this rouliad the space of all possible

computation yes and this idea of you know being at a certain place in the rouliad which corresponds to sort of a certain way of of rep of a certain set

of computations that you're representing things in terms of okay so as you expand out in the rouliad as you kind of Encompass more possible views of the

universe as you Encompass more possible kinds of computations that you can do eventually you might say that's a real win you know we're colonizing the rouliad we're we're building out more

paradigms about how to think about things and eventually you might say we we won all the way we managed to colonize the whole Riyadh okay here's the problem with that the problem is

that the notion of existence coherent existence requires some kind of specialization by the time you are the whole rouliad by the time you cover the whole rouliad in

no useful sense do you coherently exist so in other words in inches the notion of existence the notion of what we think

of as as definite existence requires this kind of specialization requires this kind of idea that we are we are not

all possible things we are the a particular set of things and that's kind of how we uh that that's kind of what what makes us have a coherent existence

if we were spread throughout the rouliad we would not there would be no coherence to the way that we work we would work in all possible ways and that wouldn't be

kind of a notion of identity we wouldn't have this notion of kind of uh uh of of of coherent identity I am geographically located somewhere

exactly precisely in the rouliad therefore I am physical space you're in a certain place in real space and if if you uh if you

are sufficiently spread out you are no longer coherent and you no longer have I mean in in the in our perception of what it means to exist and to have experience

it doesn't happen now so therefore so to exist means to be computationally bonded I think so to exist in the way that we think of ourselves as existing yes the

very active existence is like operating in this place that's computationally reducible so this is just giant mess of things going on that you can't possibly predict but nevertheless because of your

limitations you have an imperative of like what is it an imperative or a skill set to simplify or an ignorance a sufficient love okay so the thing which is not

obvious is that you are taking a slice of all this complexity just like we have all of these molecules bouncing around in the room but all we notice is you know the the the the kind of the flow of

the air or the pressure of the air we're just noticing these particular things and the the big interesting thing is that there are rules there are laws that

govern those big things that we we observe yeah so it's not obvious that's amazing because it doesn't feel like it's a slice yeah well right not a slice well it's like uh it's like an

abstraction yes but I mean the fact that the gas laws work that we can describe pressure volume Etc et cetera et cetera we don't have to go down to the level of

talking about individual molecules that is a non-trivial fact and and here's the thing that I sort of exciting thing as far as I'm concerned the fact that there

are certain aspects of the universe so you know we think space is made ultimately these atoms of space and these typographs and so on and we think

that uh but we nevertheless perceive the universe at a large scale to be like continuous space and so on um we uh in quantum mechanics we think

that there are these many threads of time these many threads of History yet we kind of span so so you know in in quantum mechanics in our models of

physics there are these time is not a single thread time breaks into many threads they Branch they merge and but we we are part of that branching merging

Universe right and so our brains are also branching and merging and so when we perceive the universe we are branching brains perceiving a branching

Universe yeah and so the fact that the claim that we exp we believe that we are persistent in time we have this single thread of experience that's the

statement that somehow we managed to aggregate together those separate threads of time that are separated in in the operation of in the fundamental operation of the universe so just as in

space we're averaging over some big region of space and we're looking at many many of the aggregate effects of many atoms of space so similarly in what we call branchial space the space of

these these Quantum branches we are effectively averaging over many different branches of possible of histories of the universe and so in in

thermodynamics we're averaging over many configurations of you know many many possible positions of molecules yeah so what what we see here is so the question is when you do that averaging for space

What are the aggregate laws of space when you do that averaging of a branchial space What are the aggregate laws a branchial space when you do that averaging over the molecules and so on

what are the aggregate laws you get and this is this is the thing that I I think is just amazingly amazingly neat that there are aggregate laws at all well yes

but the question is what are those aggregate laws so the answer is for space the aggregate laws Einstein's equations for Gravity for the structure of space-time for branchial space the

aggregate laws are the laws of quantum mechanics and for uh the case of of molecules and things the aggregate laws are basically the second law of thermodynamics

and so the um though that's the and the things that follow from the second world of thermodynamics and so what that means is that the three great theories of 20th century physics which are basically

general relativity the theory of gravity uh quantum mechanics and statistical mechanics which is what kind of grows out of the second row of thermodynamics all three of the great theories of 20th

century physics are the result of this interplay between computational irreducibility and the computational boundedness of observers and you know

for me this is really neat because it means that all three of these laws are derivable so we used to think that for example Einstein's equations were just sort of a wheel in feature of our

universe that they could be the universe might be that way it might not be that way quantum mechanics is just like well it just happens to be that way and the second law people kind of thought well

maybe it is derivable okay what turns out to be the case is that all three of the fundamental principles of physics are derivable but they're not derivable just from mathematics they require or

just from some kind of logical computation they require one more thing they require that the Observer that the thing that is sampling the way the

universe works is an observer who has these characteristics of computational boundedness of belief and persistence and time and so that that means that it is the nature of the Observer

you know the rough nature of the Observer not the details of oh we got two eyes and we've observed photons of this frequency and so on uh but the the the the the kind of the very coarse

features of the Observer um then imply these very precise facts about physics and it's it's I think it's amazing so if we just look at the actual

experience of the Observer that we experience this reality it seems real to us and you're saying because of our bonded nature it's actually all an illusion

it's a simplification yeah it's a simplification right what's what's you don't think a simplification is an illusion no I mean it's it's well I don't know I

mean underneath uh okay that's an interesting question um what's real and that relates to the whole question of why does the universe

exist and um you know what is the difference between reality and a mere representation of what's going on yes we

experience the representation yes but the the question of so so one question is uh you know why is there a thing

which we can experience that way and the answer is because this rouliad object which is this entangled limit of

all possible computations there is no choice about it it has to exist it has to there has to be such a thing it is in in the same sense that you know two plus

two if you define what two is and you plot pluses and so on two plus two has to equal four similarly this really add this limit of all possible computations

just has to be a thing you that is once you have the idea of computation you inevitably have the rule yeah you're gonna have to have a rule yeah yeah right and and what's important about it

there's just one of it it's it's it's just this unique object and that unique object necessarily exists and then the

question is what uh and then we uh once once you know that we are sort of embedded in that and taking samples of it that it's sort of inevitable that

there is this thing that we can perceive that is you know the the our perception of kind of physical reality necessarily is that way given that we

are observers with the characteristics we have so in other words the fact that the fact that the Universe exists is it's actually it's almost like it's you know

to think about it almost theologically so to speak and I I really it's it's funny because a lot of the questions about the existence of the universe and so on they they transcend what kind of

the science of the last few hundred years has really been concerned with the science of the last few hundred years hasn't thought it could talk about questions like that yeah um and uh but I think it's kind of and

so a lot of the kind of arguments of you know does God exist you know is it obvious that I think it in some sense in some representation it's sort of more

more obvious that uh that something sort of bigger than us exists than that we exist and we are you know our existence and as observers the way we are is sort

of a contingent thing about the universe and it's more inevitable that the whole the whole universe kind of the whole set of all possibilities exists but but this question about you

know is is it real or is it an illusion you know all we know is our experience and so the fact that well our experience

is this absolutely microscopic piece of sample of the rouliad and we're um and and you know there's this this point about you know we might sample

more and more of the rouliad we might learn more and more about we might learn you know like like different areas of physics like Quantum Mechanics for example the fact that it it was

discovered I think is closely related to the fact that electronic amplifiers were invented that allowed you to take a small effect and amplify it up which hadn't been possible before you know microscopes have been invented that

magnify things and so on but they're you know having a very small effect and being able to magnify it was sort of a new thing that allowed one to see a different sort of aspect of the universe

and let one discover this kind of thing so you know we can expect that in the rouliad they're an infinite collection of new things we can discover there's there's in fact computational energy

ability kind of guarantees that there will be an infinite collection of kind of you know pockets of reducibility that can be discovered

boy would it be fun to take a walk down the woolly ad and see what kind of stuff we find there you write about alien intelligences yes I mean just these worlds yes well

computation problem with these worlds is that we can't talk to them yes and and you know the thing is what I've kind of spent a lot of time doing is just

studying computational systems seeing what they do what I now call ruleology kind of just the study of rules yeah and what they do you know you can kind of easily jump somewhere else in the

rouliad and start seeing what do these rules do yeah and what you says they they just they do what they do and there's no human connection so to speak because you think you know some people

are able to uh uh communicate with animals do you think you can become a Whisperer of these trying that's what I've spent some part

of my life have you have you heard and well are you at the risk of losing your mind sort of my favorite science discovery is this fact that these very simple

programs can produce very complicated Behavior yeah and that and that fact is kind of in a sense a whispering of something out in the computational

universe that we didn't really know was there before I mean it's you know I it's like you know back in the 1980s I was doing a bunch of work with some very

very good mathematicians and they were like trying to pick away you know can we figure out what's going on in these computational systems and they they basically said look the math we have just doesn't get anywhere with this

we're stuck there's nothing to say we have nothing to say and you know in a sense perhaps my main achievement at that time was to realize that the very

fact that the the good mathematicians had nothing to say was itself a very interesting thing that was kind of a sort of in some sense a whispering of a

different part of the rouliad that one hadn't you know one wasn't was not accessible from what we knew in MA and so on does it make you sad that you're

exploring some these gigantic ideas and it feels like we're on the verge of breaking through to some very interesting discoveries and yet you're just a finite being

that's going to die way too soon and that scan of your brain or your full body kind of shows that you're yeah it's just a bunch of meat it's just a bunch of meat

um yeah does that make you make you a little sad kind of shy I mean I kind of like to see how all this stuff works out but I think the thing to realize you know it's an interesting sort of thought

experiment you know you you say okay you know let's assume we can get cryonics to work and one day it will that will be one of these things that's kind of like chat gbt one day somebody will figure

out you know how to get water from zero degrees Centigrade down to you know minus 44 or something without it expanding and you know cryonics will be solved and you'll be able to like just

uh you know put a pause in so to speak and you know uh kind of reappear 100 years later or something and the thing though that I've kind of increasingly

realized is that in a sense this this whole question of kind of the the um sort of one is embedded in a certain moment in in time and you know kind of

the things we care about now the things I care about now for example had I lived you know 500 years ago many of the things I care about now it's like that's totally bizarre I mean nobody would care

about that it's not even the thing one thinks about in the future the things that most people will think about you know one will be a strange relic of thinking

about you know the kind of you know it might be or might have been a theologian thinking about you know how many angels fit on the head of a pin or something and that might have been the you know

the big intellectual thing so I think it's a it's a um uh but yeah it's a it's a you know it's one of these things where particularly you know I've had the

I don't know good or bad fortune I'm not sure I think it's it's a mixed thing that I've you know I've invented a bunch of things which I kind of can I think

see well enough what's going to happen that you know in 50 years 100 years whatever assuming the world doesn't exterminate itself so to speak you know

these are things that will be sort of centrally important to what's going on and it's kind of both it's both a good thing and a bad thing in terms of the passage of one's life I mean it's kind

of like if everything I'd figured out was like okay I figured it out when I was 25 years old and everybody says it's great and we're done and it's like okay but I'm gonna live another how many

years and that's kind of it's all downhill from there in a sense it's it's better in some sense to to be able to you know there's there's it sort of

keeps things interesting that you know why I can see you know a lot of these things I mean it's kind of I I didn't expect you know chat GPT I didn't expect the kind of uh the sort of opening up of

this idea of computation and computational language that's been made possible by this I didn't expect that this is this is ahead of schedule so to speak um you know even though the sort of the

the big kind of flowering of that stuff I'd sort of been assuming was another 50 years away so if it turns out it's a lot less time that's pretty cool because you

know I'll hopefully get to see it so to speak rather than well I I think I speak for a very very large number of people in saying that I

hope you stick around for a long time to come you've had so many interesting ideas you've created so many interesting systems over the years and I can't see

now that GPT and language models broke up in the world even more I can't wait to see uh you at the Forefront of this development what you what you do and uh yeah I've been a fan of yours

like I've told you many many times since the very beginning I'm deeply grateful that you wrote a new kind of science that you explored this mystery of cellular automata and inspired this one

little kid in me uh to to pursue artificial intelligence and all this beautiful world so soon thank you so much it's a huge honor to talk to you to to just be able to pick your mind and to

explore all these ideas with you and please keep going and I can't wait to see what you come up with next and thank you for talking today we hit it thanks we went past midnight we only did uh four and a half hours I mean we could

probably go for four more but we'll save that till next time to uh this is round number four well I'm sure talk many more times thank you so much my pleasure thanks for listening to this

conversation with Stephen Wolfram to support this podcast please check out our sponsors in the description and now let me leave you some words from George Cantor

the essence of mathematics lies in its freedom thank you for listening and hope to see you next time