Cog Sci Special Guest Lectures: Dr. David Kirsh

all right so i guess if we have any questions we can ask sure cool uh yeah well i suppose we could start

now uh i'm sure everyone knows who dr kirsch is i suppose i'll do a little introduction

uh he is a professor at ucsd and was the previous chair of the cogs department and he is the head of the uh

interactive cognition lab and he'll be talking about his career path and his current research topics today well thank you um i'll go quickly over

some things that amuse me uh and i hope you will find them amused so i'm going to tell you who i was as an undergraduate uh and then a graduate and so on

and um you can ask any questions you like because i think there uh was a difference in how we proceeded in shaping a life so um i'm not at the end of my career but i'm in the latter

part of my career and um you know i have the benefit of hindsight so i went to the university of toronto i double majored in economics

and philosophy um after my got my va um it was after all those hippie days well late hippie days um i traveled for a few

years i was able to get a job with somebody uh whom i met in a conference on paris psychology that's you know esp and things like that

telepathy um and i got hired to go to india with um a land rover and do a study on

mystics and holy men in india so i was very excited naturally about that and i went down to the um person who hired me who was in

switzerland just outside of geneva i won't tell you about him but it's exceptionally odd individual um i traveled then well right after university i went off

to greece um and italy and then later went to these other things and met this person and so on when i went to india um i went overland

so i spent six months traveling through uh eastern turkey iran afghanistan pakistan and india and it was a huge influence on my life

as you can guess i made a i i was sent to london to buy a land rover i got a long wheelbase land rover with jerry cans and rigged it all up with all

sorts of things a library in the back and water and lots of equipment so i could i repaired a car learned how to repair it and then went off and uh

did that on the way and then parked the car in india and wandered around barefoot mostly for the better part of a year

my goals during that period were very much to understand mind and its place in nature that's the kind of thing you know various sorts of drugs had become

extremely popular we went to all the rock festivals those were those days right golden days and people were asking questions about yoga and other things like that and i wanted to know what the great

teachers around the world had to say about that both in sufi kabbalah christianity um buddhist masters i'm at the dalai lama this is this is a monk who was a

stand who was standing he never sat down or lay down for 12 years he was 12 years at it this is just a generic buddhist monk i met lots of hindu gurus and that was my job and also to meet

sadhus wandering holy men that was my job so i was lucky fellow um i came back to the university of toronto um after a few years as i say i wasn't

sure if i could um be able to do academic work after being out for so long but i found i had a lot of perseverance and was able to take it up again

i got a master's in philosophy and i had to do a lot of course work as you can imagine there was no thesis there but i did work in all these different areas philosophy of

mine philosophy of science history of philosophy ethics game theory cs purse and so on i then

got accepted to go to oxford to do work in philosophy um i was interested in philosophy of cognitive science but it was just starting then so um i you know paid my dues um this is

the new comfort freeze room a beautiful place to work i used to sometimes work right over there and uh there was in front of me oh maybe it was over there yes that looks more familiar um and

right in front of me were the erasmus operae from the 1600s these are the uh texts of the of the original of erasmus and there was also the descartes

descartes set both in french and latin i didn't uh read them but there they were those each one of those books today would be worth i don't know how many how much they were there they were right in front of you what a very

inspirational um as an a graduate student in philosophy i had to present an essay every two weeks and because i took a long time

for six years that's a lot of essays and the topics were in philosophy of mind these were my chosen topics philosophy of mind philosophy of language philosophy logic philosophy of science

mr purse again cs purse and i took a number of other field topics in other areas like linguistics cognitive psychology ethology and artificial intelligence my topic was on

foundations of cognitive science and it covered the theories of fodor and bennett chomsky newell and simon artificial intelligence and my own take on things

after that i was so lucky you can imagine how my parents were delighted to think that their boy wouldn't have to try to get a job in a philosophy department because it's

harder to come by but i managed to get to the computer science department at mit in the artificial intelligence lab and uh

i was working with patrick winston who just very recently died unfortunately very nice man i had an office beside marvin minsky who was one of the founders of artificial

intelligence and these are the kind of mobots little robots that rod brooks was building at the time we loved i love the ai lab i was one of the 20 of the people who would uh stay all

night and leave about between six and nine in the morning so uh 20 of the people would be there all night anyway when i was at mit in artificial

intelligence um i my primary research areas were about planning causal reasoning functional reasoning i learned machine learning at the time i went to i

was so lucky again to be involved in a faculty discussion group i was a postdoc there for two years and then that became my first faculty job on the research faculty though

not not a tenure track faculty job and um i did work in in these areas um then five years after that the ucsd cognitive science department was just starting and i have to say

there were 600 candidates for three spots open and uh for externals and uh marty cerrino and giovanni and me we were the lucky ones and

i can tell you with confidence there were at least 200 people who looked better on paper than me so there is a lot of luck in one's career path a lot of luck i can only say that you

can try to optimize things by being socially easy going or bright on your feet and that matters a huge deal and you can network effectively by trying

to be in the right place but um there's just a lot of luck i can only say that there are many people out there who would deserve well who certainly would have been

at least as deserving and many who would be more deserving of a spot i would have thought so um uh that we started the department in 1989 and i came a little a few months after

it started and uh that was the beginning we had great times wonderful times and then we moved into the new building you just saw there which is a very old building

now um and my my research topics now they've changed over the years but i've always been interested in interactivity interactivity in hci lately interactivity and architecture interactivity

and creativity interactivity and thinking and thinking in our senses how do we use like vision to think in vision and how do we think and taste

and how do we think and smell and how do we think and touch uh so a lot of that has to do with this 4e cognition uh embedded extended embodied and

inactive but it has a lot to do with some other things i've always been interested in creativity uh in architecture i've been

just in generative design but i spent i was also very lucky to hook up with uh the choreographer the royal ballet

and he has a dynamic um private group called random dance and i worked for him with him for i don't know he likes to say collaborated which is very flattering

uh for i don't know five or six years together we worked on many pieces and um i would go to london and capture them and uh then come back and try to see if there was something

cognitively interesting and you know little theories came out about a body cognition for instance thinking with our body and lately i've been working on the knowledge of masters and embodied theory of what masters know

and i i'm currently the president of neuroscience for architecture which is a uh i mean the academy of neuroscience or architecture which is a group trying to um advance the field of

architecture by bringing in more science topics and especially neuroscience um i'm happy to take a few questions now and then

i would go on and hope to stimulate you by giving you an example of a few of these areas but not all of them i have a question about the choreography so

you said that you like watched what the choreographers did and you made theories about it like did you just

like look at the how the dance moves like represent things or um much of modern choreography has no

representational content per se it's it's the movement represents the movement but um sometimes it does you know uh and people always like to project emotions

so when when when when a male dances with a female or a male a male or female female there's often uh the belief that that's revealing relationships of some sort

um but when i say i studied it i would go out with uh it started at ucsd so uh wayne mcgregor he's the choreographer i'm speaking out

he uh came to usd to see if he wanted to be the first innovator in residence of sixth college and he went around and looked at they

met 25 faculty because he liked science and he'd done things with scientists in in the uk before like a surgeon or other things and he would go in and see how that happens and that would be

inspiration for him so he was hoping for some inspiration for his choreography by meeting scientists and working with them in some way and he spoke to 25 and we hit it off and

then he brought me to london and he had a workshop i guess i was a good performer in the workshop and then we talked about what he might do at ucsd if he came out and i said oh i'll make a class and i'll study your

method and um we can uh explore what what it would mean to understand better the nature of creativity and

thinking with the body and so on and so he said okay and he came out with his company and while he was here they were here for three weeks i made a class of 25 students

and we would go to um uh mandeville auditorium and then later the black cube inside calit where they were

built making the dance uh and and uh i would set up six video cameras all around on high poles automatically collecting data and cameras on the top on the ceiling

looking down and the students would discuss the students we had a coding method an ethnographic coding method to code and there would be a team of

three students on each dancer and they would take turns so for the period in which the dancers were working that would be five to five hours or so there would always be a student there

every day so that's a lot of work on the part of students and a lot of coordinating the student the dancer must never be uncovered so they had to work out their schedule such that they do that we're able to

to cover them and uh i covered wayne and uh the uh assistant choreographer and um i uh and then at the end of the day we would

also take videos of the students would come and interview their their dancers and have them dance the answers to the questions they had

because dancers must make a lot of choreographic decisions so if i tell you imagine this is a pyramid with a big opening in front and i'd like you to step at us a little you know you've got to step over

something and i say step into that now depending on what the size the pyramid is and so on that they have in their mind they are going to do it in a dancerly way

so they have a lot of decisions to make how they're going to why did you do it this way why not that way and so on and so dancers are happy to talk but often

what they say isn't as revealing as when they explain specifically how i considered doing this and i considered doing this and the reason i did this was because you know

and so it's much more interesting to see the way they express themselves in their body as answers to questions we had all this on video and i interviewed wayne myself

for uh often an hour in the in the morning and an hour in the evening uh normally he gives one 45-minute interview to a person and i had like 22 hours of interviews so we knew each other very

well by then and then we and then in the class later and in subsequent years as well every year i'd go to london and set up the cameras and come back with

all these video data and we would find phenomena of interest so as an example one phenomenon of interest would be

how he used sound so to help communicate to the dancers what he wanted to do so he would say like da dada

as a you know way of communicating so you get that kind of emphasis structure and the emphasis structure in time can communicate what he wants a particular shape

now the majority of uh dance moves are generated by the dancers in response to choreographic problems that he would give them so they have a method of creativity

and most modern choreographers are using the dancers in some way as sort of idea pumps coming up with lots of things and then they look at it and they they like this they

don't like that they mix this they edit it they expand it they use something what somebody else did they might have somebody else take the ideas that the one dancer had so there's lots of ways of generating

creativity in i mean novel forms of movement and interesting ideas that don't come from the choreographers saying this is what it is like a script in music it doesn't form like that most of the

ones that i'm familiar with anyway so that's so you have to have a coding language in ethnography it is anything but just sitting there and coming up with some ideas that sounds really interesting and a lot

of fun i'll tell you a lot of fun it's again one of those things where you're lucky i'll show you something in a minute but um that that's uh i might be starting up to doing some other work with him

but right now we're uh i'm with the architects so that's different stuff um does anybody have any comments or you know questions about my little past are those pictures you

used yourself what's those for me wow are you amused yeah i wasn't expecting it but i guess maybe

that's the time period well you know it's always open people can do all sorts of things the world is still big parts of the world are more dangerous to go to um when i was in afghanistan

though we couldn't go in the north we had to go in the south um so there are really two major roads and uh we took the south road the north road

the the middle road is broken the north road you would be there'd be a high likelihood that someone would steal your car and take all your possessions and tell you to try

to find your own way home which would be quite dangerous up there um it's it's you know it's nomadic it's it's barren it's afghanistan

so um there's danger uh many places but i was machine gunned for i mean i was shot at in some places but um you know

[Music] you can have a good time it it's it's a lot of fun it's exciting and uh but i would be more cautious now about going in certain places

um being an american going in certain parts of the islamic world might be a little unwise not quite the same but still there's lots of places you can go to that would be just

absolutely fascinating and interesting and there's no reason you have to get your career going like that that's the whole point look at all that sort of figuring

out what what what do i want in life who who do i want to be who am i what and and and i know that the world is much more pragmatic now but

it's always been pragmatic you know you have to decide for yourselves um that's what i think the lesson of my past is is to try to you know find out a little bit about yourself

so would you say that time was like really beneficial to what you did in the future or figuring out what you wanted to do well it shapes how you do things

i mean of course it shapes what you're interested in but everybody comes interested in things however so i was very interested in in states of consciousness

uh when i left uh university after four years um but um uh and in india you got a big opportunity and in all these religious practices it's a

lot about this sort of inner life uh inner consciousness but um i considered doing that in graduate school

but i didn't go to one of the places where i was accepted that i might have done it and um you know the traditions of philosophy gave rise to their own way of a certain

way of thinking and um i enjoyed that and so when i went to oxford although i was still very interested in the body mind or some people call it the

mind-body relationship um and always was by the end of my thesis i was comfortable in

understanding the body-mind relationship for myself in terms of its relation to computation and

beliefs and desires the pure feel of consciousness always resists understanding but but there are there are approaches toward it so i i'm more at peace with that

um yeah so you'll always have things that interest you in approaches that you're going to take and where you are shapes what you do so

i've always tried to find the um most interesting people i could ever find in the world and lucky enough to meet them and and that's part of my life is meeting those

very interesting people um you can always do that many interesting people always around um so let me tell you a little bit about um

one thing that i work i've been working on and continue to work on this is uh about interactivity which is a very central topic to me it's a new theory of interactivity

so um what is the nature of interaction in here now you all know what people normally talk about interaction

uh mean when they're talking about interactivity in computers like human computer interaction you move a mouse or you use a wand or you do something and something changes in the digital

world and you have to somehow control the thing that's changing in the digital world so it's changing in response to what you're doing and that is crossing this digital this physical digital divide

and that's interaction but of course there's physical interaction when i hit the door and so on and when i walk all right so there's another kind of interaction it's not quite the same as physical

digital but it's it's another kind of interaction and there's lessons from each that we can learn about the nature of interaction well i started thinking about interaction in architecture because

i was lucky again to be able to go off to uh the best school of architecture in the world the bartlett school of architecture in london and um i was there and i still am there i

guess it's four or five years now so um you say well what is the nature of interact there's a lot of social interaction here and people are working and they might be working

together off to the side they're working together they're mediated by various sorts of things here people play music there's pool all sorts of interactions so how are we supposed to understand interaction

well let's contrast some kinds of interaction so here's physical interaction here's sensor interaction the person is not looking for the thing

to they're not like going they might if it doesn't work but otherwise they just walk in the doors open but they are interacting even though they might not even be know that they're interacting nonetheless

they are interacting it's like you know bumping into the wall or you may not know that you're interacting with all the things you are as you're sitting you forget you're interacting with the chair

all right so there's sensor interaction here we have classical mouse interaction here we have another kind of interaction uh here we have yet another kind imagine they're all talking to each other

mediated by the phones uh here we have all the interactions that happen where we have sensors and the different kinds of input devices

all over the place and we might have heart rate monitors on our on our watch and uh other kinds of uh pressure sensors sweat sensors so this

is the near world right a lot of people have this now not that many but more we'll be having them soon as we have much more quantitative health and quantitative

uh and then those on body and near body and in body sensors and actuators they're going to be starting to talk into things right you

know that's not long off so contrast that with this so how do what what's the nature of this interaction this person's uh looking up but is

looking interaction can't be they don't cause any change in the world this person is i don't know they must be moving i guess but anyway this is really a kind of interaction but what is that interaction

and what sort of interface does architecture represent so when we look at when we look at all these other things we have um interfaces so you can't have

interaction without an interface but what what is the nature of that interface so here we have insta interactive installations and we say well you can stick your head

into it a nook what about this can you hear that yeah i could hear it thank you okay so what what is going on here this is a very

natural kind of thing it's an architectural space and and about whoa what is that interaction it's not just social interaction they're

interacting with the lines on the you know they're interacting with the shapes and the lines and it's whoa it's a strange sort of thing and what sort of interface is that

so what sort of interaction is this when this is historic you know been around for a lot of times still a lot of places where the architecture looks like this and what what sort of interface

is this what sort of interaction is that now these questions as obvious as they are they haven't been well asked so it's a wide open territory to talk about and of course they make a

difference so here what sort of interface and interaction is this this is joint activity where people are

doing something together coordinated and their bodies and everything is spatiotemporal what are we supposed to make of that now you may be wondering what kind of

question is he asking well that's fine because i just want to show you that interaction is not as simple as people might think when they think

about a mouse and a computer so um what about this now rhythm is hugely influential in social activity

and and the rhythm mediates our coordination it helps us coordinate these people are much more efficient because they're doing it in that

rhythmic pattern wonderful uh youtube on this i mean it was a a big product it's only 10 minutes long but people

when they walk when they exercise when they have to do manual labor rhythm makes it much easier you can work harder longer if you get the rhythm

right and you can coordinate your timing so much better with rhythm so where's the interface is the music what is it how do we even

think about it what about these people playing together it's hard to imagine a robot in place of one of these people why that would be human robot interaction

so we know there's such a thing as human robot interaction we know this human human interaction now this human interaction is mediated by these objects but why would it be hard to put a person a robot in there

because we know that these people are sharing experiences they're sharing emotions they're sharing ideas of what they like and don't like that's valuation

they're sharing their own sensory motor engagement of the engineering space all of these are

uh about the social niche that they're interacting in and i believe that the next phase of human computer interaction

will involve interacting based on sharing spaces these are these spaces we have to understand there's a lot of work doing in social neuroscience now on these things shared emotion not so

much on shared valuation but on shared sensory motor maybe shared experience you know with mirror neurons and stuff so there's i mean this is very ripe it's what's happening

we haven't transferred it to architecture yet or to hci so hci can benefit by understanding architects don't think about it this way they just design in this way here's an

architectural theory of interaction so we want to design to facilitate this sort of thing and we wanted we want to design so that we could improve joint activity make the complexity of

social interaction more manageable take greater experiential delight all these things are connected to a theory of architecture and a theory of

interaction it seems to me so this is a different view of what interaction is and it's a social ecological view that

has bodies different from thinking of it the other way now here are the the ways that i distinguish there was the classic way you know where you have an input the computer takes the

input the computer puts an output you get it and it goes around in a circle okay so the output comes on a monitor the input comes in through an input device like a mouse

it's nice and simple like that then what about when they're like 12 sensors in the room including video cameras and there are multiple people around

now you can't think of it like this the inputs you don't you're not like moving it you're walking in front of the camera but the system can interact with i mean the system can do something to it so you're having an effect on the camera

the camera's having an effect on whatever the computation is the computation is having an effect back in the environment in some way maybe it makes a noise maybe it just displays your screen

of you walking past maybe it invites you to buy something so now we have this much more network notion and it's not the same as this notion so this is the one that

people are working with now ubiquitous computing and and now here we have this other one where people have situation awareness they engage in joint activity

and so on it's a more ecological social and embodied view not like this it's something like that how are we going to make sense and how's it going to change life you can be sure it will and one of the

ways is you can look who's adapting to whom now here uh the computer responds but we adapt to whatever the computer does

you can change the programs you can make the programs more complicated but when you look at humans the uh adaptation is very much

uh we're shaping artifacts and we're we are adapting but it it's not exactly the same here if we put ai in here the ai adapts to humans they

learn about us because they're context aware we may not even know about them and here everything there people are co-adapting

so ultimately the ai could co-adapt but initially it's trying to adapt to us but here everybody's engaged in this joint joint joint joint and that's co-adaptation again very much

coming very much in the view of what's happening and here's an architectural world just to finish with that so does anybody have any questions about that

so uh i'm kind of confused about the like in so like the architectural interaction is that under like the co-adapting

yes so the uh well look the architecture provides an environment within which certain things happen

and what we'd like to understand is how does this three-dimensional or three-dimensional plus changing things like sound or

changing light or media how does that change the way we interact so you have to start looking at it more like a

an ecological niche we're inside the niche now a niche is a very dynamic notion you may not think it is but it's a very dynamic notion

where um the changes uh the the creatures change the very environment they're operating in which then has an effect on how they

interact themselves again with the environment so the the nature of interaction is much more complicated and that's the thing the the environment

is creating like so in architecture you might you you would design for instance to make it increase the chance that people will get together

architecture is very much about social things helping people to get together helping people to work together helping people to find their way about things

so or in a classroom design helping the people transmit information in a way that they can get it well and and so on and they might decide oh we've got to make it quiet in

here so that the person in front the acoustics are front directed rather than from the back forward and we do lots of changes in the environment to facilitate the social

interaction so it's a niche and and and uh all the people are constantly in co-adapting to each other

and the room is responding in its own way like it's uh if you're in a room that's really loud you'll get more quiet

well when you get more quiet you you know the there's a response the whole thing changes because suppose the walls are very sharp and

there's no carpet on the floor then the the sound in the room is really loud and harsh people hear it they get quieter as they get quieter there's less things coming back so i'm

interacting with the walls through my speaking and noise because i change the wall by the sound and the sound hits the wall the wall bounces it back to me and and

so i we change the whole system changes of me and the walls everything changes a little bit that's that's this that's this bigger notion of

interaction very niche three-dimensional which is not yet what we're doing in our computers okay that that makes sense it's not easy

by any means so it's it's not like that it's it's hard but it i believe it represents a new way of looking at things and so anytime somebody says there's a new way of

understanding then you know you really need lots of cases and examples and proof that something good comes for from it by by seeing the world this way

before people decide that it's useful but it's something i'm interested in right now uh and so i have this analysis and i'm just introducing it i'm not giving you all the reasons why i think we can make

these definitions as clearly as i passed now in neuroscience for architecture

we would like to understand um what's going on in the brain that affects performance in the environment or our

enjoyment of the environment or perception of beauty and so on um and uh there are some simple things so in

neuroscience there's fairly good research on the nature of circadian rhythms and where they happen in the in the brain and how they work and their sensitivity

to light well they're sensitive to different wavelengths of light so a simple very dumb i mean by by dumb i mean very simple uh uh the

thing that neuroscience can advise architects on a topic it would be well what color should the little nightlight be in your bathroom so that when you get up in the middle of the

night if you get up it won't wake you up and you'll be able to have your tinkle or have your wee and go back to the back to the bed and you'll be asleep better

than if you had a different light well that light is a particular shade of orange some kind of orange not exactly this shade but it's it's a kind of orange and um and if you put blue in which is the

majority of night lights are blue they wake you up so that's a little piece of neuroscience right now a lot of lighting questions could have neuroscience bases

well here's a topic that i'm working on with an honors student in the quagside department undergraduate she's interested in uh mild cognitive

impairment and alzheimer's disease and um so we wanted to explore some of the neuropsychological uh

data that could advi be useful for architects to know when designing better uh facilities

for people with mild cognitive impairment or are alzheimer's so for instance people lose their way

so should we um color code their area should we have a picture of their child or or something else on their doors so

they recognize their door or picture of themselves on their door shall we i mean obviously the easiest way would just be put something in their pocket that lets them know how to get there

keep going keep going you're almost there just a little bit on the right that would be the best way but if we don't do it digitally and we do it through architecture what what are the best ways should that

you know in a large facility should there be regions and when you enter a particular region for instance we make it like main street of an old town

or there's a saloon so we make it a western town oh this one is paris and so we have parisian stuff would they remember that better will they know how to get there

better well what effect does brightness have on memory what effect does brightness have on other performance things that we consider for instance

alertness or stress or the ability to recall the ability to recognize our level of engagement the ability to concentrate how does

brightness affect that that's something that a neuroscientist might say something about so we need to know what are the units that we use to measure alertness or

ability to concentrate or engagement so we need neuropsych units and then we have these the architectural things we could change we could change the noise level the brightness the color

the temperature these are all very primitive now these are not like big structural things right but these are things that you could imagine would be very helpful so we could say well you know what uh

maybe we should change it's it's good to keep the noise low but not too low for alertness and if you want them to remember maybe we should have

the noise at a certain things or maybe they'll remember where they'll remember where they are if it's if they're different noise levels so oh

i live in a noisy part of the building so as you start going into this direction it's noisier and this tighter direction it's quieter

or maybe you can see so many questions so this is how we could begin to engage there's not much done on this yet but it's a new field and there will be a growing field the field of greater importance

it's just at the point where it's going to take off i believe so this is an example of how one might do studies on that any questions

so if you were to like so this is a new field but say if you were going to do like a career would you work with like engineers or would you just more be like

the research well you won't find a career right now in this but if you were to uh take an interest in this

in this particular thing this is a br this would become a branch of neuroscience so one the neuroscientists are the ones who have to do the work to get closer to the architects

it's good to get a team where you work with architects to understand what the possibilities are because you know uh it's one thing to say oh uh

the more light there is the more people can stay awake or the more alert they are or the less stress or maybe the more stress i don't know but i expect you know so so let's say alertness or

engagement better to have bright rooms now older people often like less bright so it may not be obvious you know maybe they find it very stressful to have it really bright so

if they could have and maybe alzheimer's are different with respect to brightness sensitivity than younger people and so on or people without alzheimer's so there are lots of questions about that right and people could just say you

know what i want to do some psychological neuropsychological experiments on that it could do that suppose they come up with some sort of answers well so they tell the architect look you

know this area should have this much brightness that doesn't tell the architect how big to make the windows how many windows to have what to do with the lighting

there's a lot of decisions to be made architecturally now if you're the kind who you know like that kind of design questions then you

might like to work with a designer because an architect because maybe in fact it's a more nuanced answer than just this

brightness so maybe it depends where it's spread like it shades off so it's quiet high up but it's tasked right down here what does brightness even mean

so we don't necessarily have you can you can experimentally we make simple decisions but but the reality is more complicated so if you're the kind of person who's interested in design

well this would be a good area because you're going to be doing something other people aren't and it is definitely going to be important in the next 15 or 20 years so if that's the kind of thing you do you don't have to worry about an

engineer who what you know in this case architects yes or interior designers but it would be primarily working from cogsi you people would be going into

a kind of neuroscience of this or the cognitive science of it it doesn't have to be exaggerate the neuroscience it could be psychological experiments like we have units and we have units

over here so we're looking for connections between them

any other questions okay i'll show you a little bit uh some work i did on um interactivity and creativity so look at this

right a lot better yeah so we have chance encounters can yield extraordinary insight and um

that's that's that was part of what my uh uh topic here was was exploring how chance is involved in uh in in so much of

uh creativity so but without physical interaction there would be no none of these chance encounters internal chance i didn't consider the nature of internal chance we get

the external chance is really the surprising thing that happens uh now the outcome isn't predictable because it's very hard to do the transformation inside

so we do it outside and we see certain things because it's easier to do things outside that are especially that are complicated the outside takes care of the complication it was just a transformation but to

simulate that transformation would be very hard i say look you see my hand simulate what my hand would look like when i do that you can't do it it's just too hard too many things are occluding

and and it's just too hard but i can trivially do it right so uh the outside does all sorts of wonderful things for us and we have to figure out how we use the

outside to do those wonderful things so interaction with the outside isn't quite random but it's not a move in the space we

might have been thinking of in this particular case and we we can do things outside that we can't reliably do inside our heads and that's the point of this the moral

of this particular example so i looked at this problem and i said look there's a lot of different ways that uh creativity and movement emerges

through interactive cognition we have cognition that is you know mediated like that piece we just saw where he makes the movement of the piece

we have notation that we can move around and do things in notation when we write or when we compose with notation and so on and we can have body mediated

cognition like i'm doing right now is body heated but but i could be looking at my hand as a three-dimensional thing or i could be feeling it on the inside

and that's another thing i could use to learn about something so um i i'm not going to give this talk here but i was looking at randomness how it helps create

creativity now um so let's just look at it often creativity people think of creativity as something that goes on in your head like it's almost like inner speech

you're talking to yourself and i say look i'd like you to give me a sentence that describes something well then you're going to be working with words in your head how they

come up into your head i don't know but but sooner or later you get this explicit thing or we ask you to build something and and yeah you might

think of it all in your head unless you know you're sculpting but before you're sculpting a lot of it is abstract and and there's just stuff going on in your head

usually like inner speech well we can generalize that that the inner vehicles don't have to just be language finite vehicles they could be

kinesthetic imagery so i ask you to design a wading pool okay so now you'd want to have the feel of wading through the water

you might have that imagery a mental imagery about waiting what it feels like or i ask you something about designing something to make it beautiful you might have imagery about a setting sun

or the smell of these are all examples of imagery and we have some control over the symmetry not as good as the uh auditory imagery

so you may not be able to imagine the sound of a saxophone so well but you can imagine the sound of words very well

so you have tremendous control over the auditory icons of words but it's just auditory imagery and what's the relation of those auditory icons

to the underlying meaning well that's a mystery isn't it but we have those sounds in our head and we could have all these different kinds of vehicles

in our head that we use and now i say well what's wrong with seeing creativity as that well what's wrong is that it's not paying attention to what we do

outside so here's an example of artifact mediated interaction so suppose you're playing a game of cards gin rummy that's you know you've got to

get uh threes or four three of a kind or three in a row four of a kind or four in a row and then you win when you have them they're ten cards

and if you have the hand out like that you start moving them around and counting which ones you've got and you end up saying okay look i okay i'm gonna move that card so i'm gonna count and yeah i've got a

three okay i've got a three five six seven uh of spades and then uh okay i've got like oh two eighths three okay so i can see three eighths and

you're moving through these internal operators of moving the cards around to know what you have and then finally you get an idea of what you want to do you have an idea you have a mental

representation of your next move and you execute the move now this is the standard old account and if a robot were doing it or a computer were doing it

this is what they would do they don't ever change the cards why don't they change the cards while they're playing what good does it do they don't they

can't they're no closer to the answer they're no closer to the to making a move by moving the cards around in their hands it doesn't change it doesn't get them

closer to winning it it may help them not them it'll help us but it doesn't help them and experts by the way they don't move it because it gives hints to other people

but but not experts they move the cards around but it's pointless isn't it no it's done for cognitive reasons and when you start looking at the moving

around we change this account of what goes on we have external operators as well as internal operators so the red ones are external operators

when we move the cards and they move the cards and then that brings oh you see here's you know three of a kind so i start out with the the hand then i move it around i move it around oh there's my three of a

kind right there and so on and so uh we do that and so now our cognition is spread in an extended mind manner over the

cards in our hand and are inside of our head all right well that we know that so and now you might say well look a lot of

thinking is more like that than it is of the other way so for instance uh when somebody plays a musical instrument they're closely coupled to that

instrument aren't they it's very important for them uh to to hear their instrument and to manipulate it

so uh thinking is is he not thinking when he's playing is he thinking what do you call it i don't know why you

wouldn't call it thinking so uh just because we can think in the head that doesn't prove that's the best place to think

maybe we're better thinking uh with an instrument in our hand or notation and uh that would be my view so here we have he doesn't know exactly what's going to

happen until he does it in fact this is the whole point of this sin it depends on how the strokes come out and how much you press and so on and and so that

there's some very tight connection to the outside world there when these uh these are not crick and watson but they when they're working with this complicated model

they feel the way the model allows them to move the pieces and it there's many documented cases where a lot of insight into the nature of molecular behavior

came by playing with the three-dimensional representations of the molecules i mean look this one's square and stuff i mean it's all sorts of strange stuff

and here this is quick when with their early representation of the double helix with watson and uh apparently this is very important to their insight and discovery of the

behavior of the double helix and why it was a good answer so here we have two people working on the same thing that so through joint activity one's talking and pointing and the other

is thinking about it and replying and all of that is joint thinking and the timing matters and so on all of that is this big interactive process that affects how

fast you can think how slow what you can think it's very important so the bottom line is we often think by coordinating

internal and external entities and structures and operations activity with artifacts

and that's a nice notion of interaction uh and and now body-mediated interaction well this this baby

it they're using it to a toy and and it's definitely physically mediated so there's a lot of artifact stuff but part of what's really going on is this baby is learning about this

and learning about you know movement and feeling and the results so there's a lot of cognition going on that's mastering the nature of that thinking with your body

now what about this so this is mental abacus and he just multiplied two seven digit numbers in his well in his head

he they know how they learn how to do it on abacus so an abacus can allow you to multiply very quickly once you learn how

so clearly they do they do not say a seven-digit number is you know 6596 478. look at how long it took me just to

478. look at how long it took me just to think of it so i can't multiply that in sounds so i'm not going to use that that sound icon thing i do when i say 7

times 9 or 63 i'm using sound when i use 120 times 3 and i say 360 i'm using sound but the process by which i do it

i i might also think visually 3 times 12 i just see the answer visually but here they're seeing the answer visually encoding numbers in the

abacuses all right now why does he gesture he hasn't got the abacus in front of him so either he has to mentally stimulate if that's how he knows to multiply

he either has to mentally simulate using the abacus or he uses his hands to help him simulate the abacus

movements so here is a case where gesture is helping to carry his thought forward and the faster he gestures the faster he can think

well that's pretty strange isn't it how much of our life is like that in tetris some early work i did with paul maglio at the cognitive science when he was the first in the beginning

of the department uh uh well we eventually did it he just started out in the beginning and then six or seven years later uh he finished and we worked on this very closely

together and um what we found was that in tetris people have trouble sometimes when they're not experts uh they sometimes get confused

they're gonna make a mistake they make a mistake about a mirror image piece so this is a you know right-handed l versus a left-handed l and um

when they have a shape so the question is will this piece turn into that piece when you rotate it and because it's got it this is a sort

of receptacle well obviously not so uh they're mirror pieces so that's the kind of thing people want to know well what

we found was that people rotate them and why do they rotate them you could say oh i'm rotating it to put it in there but what if it's the wrong piece i can't be rotating it to putting in there you could say i made a mistake

maybe but maybe i'm rotating it to figure out which piece it is because when i see that it isn't the right piece then i know it's not the right piece so

in fact my action wasn't really to get it in the position to go there it was to make sure it is the right one that could go in there and that is a kind of what i call an epistemic

action an action that has to do with changing your mental state more is more important than changing the physical state it happens because we're so tightly

coupled then how fast and we found that people can physically rotate faster than they can mentally rotate so depending upon your

twitch speed you can rotate one of these uh make a make a hit the uh whatever you're using the up arrow down arrow or

you can hit them at between 80 and 120 milliseconds but to make a quarter turn in mental stimulation

takes considerably longer could take two 300 milliseconds for mental stimulation so it's faster to physically rotate faster to rotate it and look

than it is to mentally rotate and if you make a mistake and it's the wrong piece and you find whoa oh this isn't going to work here the cost of that is more than offset by

discovering it that it's the wrong piece early so there's a case where we show how much does it cost to do it outside how much

does it cost to do it inside it's better to do it inside so uh i see i'm running late on time i just wanna and now here is what people

do to invent use a choreography

so they tell him i want

sort of an exercise all right well done noah so now here is

um he would have been given an exercise so they would have been told either they were in a banquet hall at some point or they were told imagine you're in a banquet hall and you saw that there were those big tables

that fold up in two and uh he took chairs and stuff and so he used the imagery and the behavior of those things and then he made the chair get bigger and he sliced

it in half he did stuff you can't do in real life but he also did stuff you could do in real life and he used the uh imagery as what he had in mind when he was doing

particular actions so he was closing the tabletop and another time he was sticking his foot through the chair to cut it and open and then he kicked it to stretch it and

then he pulled off a leg all of those things were movement ideas that he got which would be far more interesting movement ideas than if

you were just on the dance floor trying to do your dance to music you know how terrible we are in terms of inventing new movements so you need help to invent new movements

so this is a case where we have using mental imagery to invent new movements interactively and we're working against our mental imagery i have lots more to say but i don't want

to say it because uh it's time now so a lot of our thinking is done by coordinating coordinating internal uh imagery or internal

ideas and bodily activity we can call it a day there uh oh then there's notation mediated there's so many ways of mediating but i will not uh why don't we just stop

there and we the rest will all be questions uh do you have a question do you think that like this interaction there's like

something special uh about humans that they could do it or do you think it's like all the same between like different animals

well it's a great question um so we can do more things with our minds

then the animals can that increases the way we interact so imagine three lines i saw this in in a game park one time we were in a land rover up the

side of the hill and there were three lions one was close to us and there was another lion on the far side there was a third lie and the fourth line was hanging around doing nothing

and there was a a buck a deer of some sort down in the valley moving around lionesses right they're the ones who really hunt

and they're moving around and they're sort of triangulating now those lions i don't know if this is true but i'm sure those lions

have the ability to project the direction that the deer would go and also to know that if they ran they could get the deer over there and that the other lion would come and

could come nearby so i think they could have drawn lines in their head in their minds where they could know where the other lines would be and where the deer would be and so because they're trying to shape it

so that they can get the deer right and um now that's a that's a visual projection capacity which i'm extremely interested in

but humans can do it in spades what we can do out is just amazing in terms of our capacity for visual um projection i i if i say to you

i want you to imagine a a triangle uh on a wall near you and now drag the triangle over the edge of the wall to like if it comes to an edge and now you'll see

the way the triangle bends to go around it you can do that that's projection so now we can do all that sort of thing

animals can do some things now do they interact with their projections do animals i mean for the most part animals are going to be they're not just reactive creatures

they're very active creatures and they're trying to get the world into the state they want it to be so there is of course action seeing what it is i do respond to the world the

world changes in what i do i respond again that's interactivity so of course animals are wildly interactive far more than our robots far more than

anything else that we have invented but uh there are different kinds of interactivity and so can they think in the world can they use interaction to think

and i was suggesting that some of the things they have some of the equipment that we use to think with using the world but we can also design the world

to make our thinking more powerful so if we know we have this capacity to project how more more is it more powerful to project

when the structure outside helps us than if it doesn't help us so can i project things onto an outside structure that i couldn't just imagine

in other words is projection more powerful with objects outside than it is in imagination alone that's a question i ask and if it is the case

then we want to design things on the outside that increase our powers and these things that we do design like notations and uh

various sorts of mechanical objects and all sorts of and we make mutant instruments that constrain what we do

and all of these are devices that enable us to think more powerfully

[Music] um i have a question sorry if you addressed this earlier i came in a little bit late but i'm just curious about what you think of ucsd's architecture because i feel like

it's it's not really like any place i've ever seen in the way things are laid out i don't know if you have any opinions on that well um so i was here i've been here for

a while not as long as some but more than most and i remember so you know of course the library was the big thing and then they

uh then they built the engineering section and that's sort of interesting parts of it the way it is but not that interesting but so the buildings weren't very interesting but they did a little something down at the bottom engineering

too um the buildings up until then would be pretty boring the supercomputer center wasn't very much until the extension um

but the the one of the big things they did was they landscaped it so um you only know post landscape so when they put the library in later they had to make a

giant extension to the library because as pretty as the library was and the part you see above ground it actually is really inefficient and

doesn't hold very many books so in the old days five times as many books were held in warehouses as they were in the library so you

couldn't get a book right away often you'd have to order it and it would come the next day or later that day or something depending upon the efficiency so they decided they had to make a bigger library and what they did was

they dug down and made a giant cube a giant rectanguloid all right but when you stand up on the on the uh library and you look

down uh what do they call it the walkway down to to the medical part of the campus please i think it's library walk just the big straight one

yeah library walk well library walk didn't exist before but it but but then they made that there and it connected the library in a straight line

over to the medical side at the same time if you go up and you look out toward engineering building engineering two is right straight to library walk so

there's right angles and you could just see somebody saying like the chancellor but not any of our chancellors but like the chancellor they sort of say

look at what i'm doing to my place i'm unifying the medical side now with the rest of the campus by this library walk the the engineering side is

now connected to the library which is in some sense the intellectual center of the campus even though no longer it is but it always was even though the the most of the

buildings aren't near it like that they're in other places and okay so the since then they begin to build some more architecture and what do i think of it some of it's

good and some of it is less good some of it is expensive and some of it is less expensive um it's not going to win any awards it's

not going to be embarrassing it's okay it's good but you know you have to look at buildings to see what buildings are happening around the world in order to decide

these are expensive structures right so i mean they make a building it could cost half a billion dollars i mean they must at least cost 125 150 million dollars

and probably they cost more so you get a fancy architect to come in and the fancy architect you know there's going to be their reputation cali tea was an interesting building it

won some awards um so that it's called qualcomm now so the qualcomm institute it was an interesting building

um i don't know what other buildings we have that have won awards uh that isn't the necessary measure of things so the new music building it's quite efficient it seems

to me to be a a successful building but i'd have to talk to the musicians inside to see if it really is um does visual arts have a new building yet

um so you know the natural science going uh yeah i haven't been very impressed you want to know the truth i haven't been very impressed with the buildings

the rainy school wasn't bad i think uh of course a raidy school was a very special building because it's overlooking the salk institute the salk

institute is treated by many as one of the in the top by most architects as in the top five buildings in the world it's a mecca if you're an architect you go to the salk

institute it was the first place to invent infinity pools it was it did so many things first and it was a wonderful building and it is still a wonderful building

and uh we have our our annual conference there for the our biannual conference for the uh architects there because at solstice winters the summer solstice

when you look this is this is a sort of path going down um and and when you uh okay let me find it i will find this for you

because um it's worth doing and then maybe we should call it because i'm taking so much of your time salk institute

uh images let me hear now here you go okay let's get this let's hope it's bill no no no no no no okay there's too many things

okay so you'll have to look at it like this i now need a new share let me go to there are you now here can you see this

i see it so this is the salk institute this is the salk institute this is the salk institute this is the salk institute you say what's all this about it was

absolutely brand new in so many ways but here when you look at this in the winter so this summer solstice the sun so the idea is this was a great

research building and you've got to you're going to infinity and it's supposed to inspire the people who work here and you know there's the outside and then there's the inside you're not going

into a big huge rectanguloid and this was done in the very early 60s and and and the the outside is concrete and it's textured you might think it's dull but it was

warm and textured and they have the salt i mean they have the antique wood and these are all areas where the professor has this little space and then it goes joins with all the rest

for the laboratory space and um and the idea of this is that people are going to walk back and forth to talk to each other and so you get going and there's a place for you to there are

little places to go outside and the light comes down into the bottom areas very creatively it just is facing infinity man and this and at the end here the reason i say

it's an infinity pool is because oh it's wet there okay well the reason it's an infinity pool is because at the end this trickles into a a little pool at the back here and there's no back and it keeps going

and the water just goes over the end so it's sort of you look here and the water goes and then you go on into the pacific it's

and the sun sets right there did somebody say wow yeah that's beautiful i don't think i've ever been is it it's

go just go and say when you go there it's right across from the school right across the street from radiates near the estanzia hotel go there they may not let you in tell them my professor told us that we have

to see the salt institute can i please see it and maybe the guard will say yes in the old days it's no problem but now they have guards you know and it's closed mostly because of covet

but when it's not try it and and uh uh you know and it's just an extraordinary building so if you were an architect you have to go visit

now who overlooks uh they that who overlooks it sorry you know there's so many things we have to put on top of here

that it gets in the way with uh brady school of it's not business but i'll call it then and they don't know what i mean

uh i mean images so now here we have rady well that looks pretty dramatic they're doing the best they can i assure you uh so they've got this it's a little

derivative i'd say but that doesn't matter it's pretty good so here's rady here's campus our our section on the uh you know this part of campus says the north part of campus is a little more

interesting but um so they have some shapes to ready it's a it's a pretty good building pretty good building but you know what what oh we've got the new building now ah the new building

looks cheap to me you want to know the truth i didn't say that the new building you know it was constrained the it was sixth and seventh college so six six college

it's so big and it's you know it's got a lot of requirements six college ucsd let's have a look at that building i'm not a master of this stuff i would love to hear a great architect a good architect even

tell me about uh so here's the new sixth college so you see it's got some of the structure this is very fashionable nowadays it's all over the world they put these slightly different i mean look at three cubes right almost and

they're just like blocks a little bit on the side they've got this stuff here well okay you know look you're an architect you're going to do it there's glass there's a little area for you to come out these are mandatory now so uh but they

have an outside theater we like that and and uh i i i'm sure it's good i haven't really been through the building so i can't say i can't say how well it plays with the light

but i mean let me show you the stator building stator building i think it's called the stator building isn't it a state of building so let's make sure we're right stater

building mit uh okay oh stata okay

so look well all right so this is frank gehry this is the computer science building right this is the computer science

builder what that what the hell uh so you know here's a computer science building and and how old is this why it's really old by now they'll probably be moving

moving to a new building in a little while so um this is the kind of thing you're having to ask about you you asked me about one building and and you asked me is it exciting well

this is what they got 15 years ago or was it 20 years ago i don't know and um you know you might think oh i think that's terrible i think it's ridiculous i think

so you also have to know how does it play on the inside and they quite like it on the inside there were some mistakes made and the architecture faculty is inside the stator building by i mean it's not a building against

but uh and you know they would have had a lot of amusement for instance there was a stairway that went to nowhere but um uh you know it so these are the kinds of things that

you're asking yourself when you say what do you think well relative to what so what do you think is it time to call

it a day does anybody have a pressing question uh does anyone else have any questions [Music] professor kirsch did you uh mention

geisel at all oh yeah i did because you know that was the that was an interesting building and it was an important building and it was it was ucsd's um you know uh

there's a reason why we use it as a uh um as a you know our not our mascot but our but our emblem and it was a building in the 60s right

so yes it was an inch it was an interesting build i think it was an interesting building maybe somebody else wouldn't but i think it's a very interesting building and i found it quite successful um in fact you in order to to encounter

the building make sure you go up to the different areas and uh go to the edge of the building so you can look out and see what it looks like from the different places

the reason it's successful to me is because of the way it gives a a vision of the entire campus so yes yes i it's a funny shape and

you know that's interesting but the consequence of the shape was i did mention that there you can't have enough books but aside from that fundamental problem with it um

and the fact that maybe you feel like you're in a some kind of weird mall on the ground floor when you're walking on top of it they could do more with that or they could have made that more interesting there's a lot about it i like and i like

the addition very much um ask yourself why do they make those jagged things why do they why do those glass jacket things shaped like that you should must figure out the answer okay when you walk by

it next time you know they're in different parts of the thing and they're light modes their function is to let the light go from outside down into the you know ground one and ground two or you know

underground one underground two where the big areas are of books we know the function why did they make those shapes like that

so you should figure that out and um uh i like it yes i like it so it was in the 60s and you know what what was uh you know

how um uh [Music] well okay important

as architecture well wait well never mind i'll just take uh

okay but what do what do we think about it uh here's what uh you know wikipedia but we don't know who

okay fine so brutalism and futurism so brutalism is um it's architecture that's really massive

and you i mean the concept of brutalism was that uh it's fascist architecture that the fascists want you to understand that you are small and inconsequential

it's the state that matters and so you are small and when you come into a big building with a large huge area with massive columns holding

it up and a big staircase going up who am i compared to this with all the stairs going up i'm nothing the state is so powerful i am powerless

compared to the state that's brutalism okay the building beats you now they say if this is brutalist in part but they say futurist too and i you know

i don't know why it's exactly futurism i i'm not the one to tell you but um i can say it's you know it's it's certainly becoming the most the most wrecking of

course it is um and uh in the opening 70 would have been built in his late 60s um he was awarded the commission well let's

see did they win anything for it did he win an award for it i mean look it's ours we're i'm telling you i you you you to appreciate it you must appreciate all the windows

look at it look it wasn't that good of him to make all those windows everywhere and in a way such that when i'm here of course these people they still can see down but they can't see up

but these people they can see everything now these people do they see everything or do they get blocked by that these are the big questions you ask and

um you know some of it maybe maybe it couldn't maybe it would be very expensive to keep building it up like this certainly would be much larger but um so down here this is what i was talking about was

this is pretty brutal down here you know it's just massive concrete and you feel it you feel the strength of the arms and stuff but anyway uh i don't know what whether they

uh it's a classic but i don't know uh whether it won any awards or not uh it's a good question i'd like to know the answer

uh he was awarded the commission but right man did he well it's not telling us if he won any awards maybe he did maybe he didn't someone could

find out for me all right and then they have this new thing down here which is what i was talking about this is clever this turned out to be quite clever we haven't time for me to tell you

stories about this building there's some really fun stories about what happened when they first opened the doors after this was built

but i'm going to save it for another day who's in charge all right thank you for coming to talk those stories sound like

they'd be interesting someday yeah uh i believe that's it uh we'll let you go but yeah thank you for coming and telling us about your

research and your your path and i'll put the recording up later for anyone that would want to see it later in its

entirety for the people that came a little later well it's always a pleasure speaking to our perfect undergraduates you you represent the future of

cognitive science so march on change the world do you have any uh parting advice for the cogs

undergraduates just find out what you [Music] will never get tired doing all right thank you uh so i guess that

concludes today uh next week we'll have have uh ellis is talking yeah so

yeah thanks for coming everyone okay thank you okay hi hi hey zerby thanks professor kirsch yeah thank you thank you

professor oh great ceo thank you professor kirsch are you still recording uh christina oh should i stop recording yeah actually i want to ask professor