Stanford XCS224U: NLU I Intro & Evolution of Natural Language Understanding, Pt. 1 I Spring 2023

welcome everyone uh this is natural language understanding it is a weird and wonderful and maybe worrying moment to be doing natural language understanding

my goal for today is just to kind of immerse Us in this moment and think about how we got here and what it's like to be doing research now and I think that'll set us up well to think about

what we're going to do in the course and how that's going to set you up to participate in this moment in AI uh in many ways in whichever ways you choose and it's an especially impactful moment

to be doing that and this is a project oriented course and I feel like we can get you all to the point where you are doing meaningful things that contribute to this ongoing moment in ways that are

going to be exciting and impactful that is the fundamental goal of the course let's now think about the current moment this is always a moment of reflection for me I started teaching this course in

2012 which I guess is ages ago now it feels recent in my lived experience but it does feel like ages ago in terms of the

content in 2012 on the first day I had a slide that looked like this I said it was an exciting time to be doing natural language understanding research I noted that there was a Resurgence of interest

in the area after a long period of people mainly focused on syntax and things like that but there was a widespread perception that nlu was you was on was poised for a

breakthrough and to have huge impact that was relating to business things and that there was a white hot job market for Stanford grads a lot of this language is coming from the fact that we

were in this moment when Siri had just launched Watson had just won on Jeopardy and we had all of these in-home devices and all the tech Giants kind of

competing on what was emerging as the field of natural language understanding let's fast forward to 2022 I did feel like I should update that in 2022 by saying this is the most exciting moment

ever as opposed to just being an exciting time but I emphasize the same things right we were on in this feeling that we had experienced a Resurgence of

interest in the area although now it was hyper intensified same thing with industry the industry interest at this point makes the stuff from 2012 look

like small potatoes systems were getting very impressive but and I maintained this here they show their weaknesses very quickly and the

core things about nlu remain far from solve so the big breakthroughs lie in the future I will say that even since 2022 it has felt like there has been an acceleration and some problems that we

used to focus on feel kind of like they're less pressing I won't say solve but they feel like we've made a lot of progress on them as a result of models getting better but all that means for me

is that there are more exciting things in the future that we can tackle even more ambitious things and you'll see that I've tried to overhaul the course to be ever more ambitious about the kind

of problems that we might take on but we do kind of live in a golden age for all of this stuff and even in 2022 I'm not sure what I would have predicted to say nothing of 2012 that we would

have these incredible models like Dolly 2 which can take you from text into these incredible images language models which will more or less be the star of the quarter for us but also models that

can take you from natural language to code and of course we are all seeing right now as we speak that the entire industry related to web search is being

reshaped around nlu Technologies so whereas this felt like a kind of Niche area of NLP when we started this course

in 2012 now it feels like the entire field of NLP certainly in in some aspects all of AI is focused on these questions of natural language

understanding which is exciting for us one more moment of reflection here you know in this course throughout the years we have used simple examples to kind of highlight the weaknesses of current

models and so a classic one for us was simply this question which U.S states

border no U.S states the idea here is that it's a simple question but it can be hard for our language Technologies because of that negation the no there

in 1980 there was a famous system called chat 80. it was a symbolic system

chat 80. it was a symbolic system representing the first major phase of research in NLP you can see the fragment of the system here and chat 80 was an incredible system in that it could

answer questions like which country bordering the Mediterranean borders a country that is bordered by a country whose population exceeds the population of India I've given you the answer here

turkey at least according to 1980s geography but if you asked chat Adie a simple question like which U.S states

border no U.S states it would just say I don't understand it was an incredibly expressive system but rigid it could do some things very deeply as you see from

the first question but things that fell outside of its capacity it would just fall down flat that was the 1980s let's fast forward 2009 around the time this course

launched wolf from alpha hit the scene and this was meant to be a kind of revolutionary language technology the website is still up and to my amazement

it still gives the following Behavior if you search for which U.S states border no U.S states it kind of just gives you

no U.S states it kind of just gives you a list of the U.S states revealing I would say that it has no capacity to understand the question posed

that was 2009 so we've gone from 1980 to 2009. okay let's go to 2020. this is the

2009. okay let's go to 2020. this is the first of the open AI models Ada which U.S states border no U.S states the

U.S states border no U.S states the answer is no and then it sort of starts to babble the U.S border is not a state border it did that for a very long time

but about Babbage this is still 2020.

the U.S states border no U.S states what is the name of the U.S state and then it really went off the deep end from there again for a very long time that was Babbage if you had seen this output well

at least for me it might have shaken my faith that this was a viable approach right but the team persisted I guess 2021 this is the Curie model which U.S state

supporter no U.S states it had a problem that it started listing things but it did say Alaska Hawaii and Puerto Rico which is an interestingly more impressive answer than the first answer

right it still has some problem understanding what it means to respond but it's looking like we're seeing some signal Da Vinci instruct beta this is 2022.

it's important I think that this is the first of the models that have instruct in the name we'll talk about that in a minute which U.S states border no U.S

states Alaska and Hawaii we've from 2020 to 2022 we have seen this astounding Leap Forward making everything before then sort of pale in comparison and then

finally text DaVinci won you know one of the new Best in Class models at least until two months ago which U.S states

border no U.S states Alaska and Hawaii are the only U.S states that border no other U.S states a very impressive

other U.S states a very impressive answer indeed and if you just think about the little history I've given a kind of microcosm of what is happening in the field

a lot of time without much progress with some hype attached and now in the last few years this kind of Rapid progress forward and you know that's just one example but

these examples multiply and we can quantify this here's another impressive case I asked the DaVinci 2 model in which year was Stanford University founded when did it enroll its first

students who is its current president and what is its mascot a complicated question indeed and it gave a fluent and factually correct answer on all counts

this is The DaVinci 3 model which was best in class until a few weeks ago and it gave exactly the same answer very impressive now in this course and for for you'll

see at the website one of the readings we've suggested for the start of the course is this classic paper by Hector Levesque called on our best behavior and the thrust of this article essentially

channeling Terry Winograd and Terry winograd's schema the idea is that we should come up with examples that will test whether models deeply understand

and in particular get past the kind of simple memorization of statistics and other things about the data they're trained on and really probe to see whether they understand what the world

is like and Levesque and winograd's technique for doing this is to pose very unlikely questions where humans have very natural answers like one of the

ones Levesque poses is could a crocodile run the Steeplechase maybe it's a question you've never thought about before but you probably have a pretty consistent answer across

this group could a crocodile run the Steeplechase here I asked another one of levesque's questions are professional baseball players allowed to glue small Wings onto their caps you could think

about that for a second The DaVinci 2 model said there is no rule against it but it is not common and that seemed like a very good answer to me at the

time when the da Vinci 3 engine came out though this started to worry me no professional baseball players are not allowed to glue small Wings onto their caps Major League Baseball has strict

rules about the appearance of players uniforms and caps and any modifications to the Caps are not allowed okay I thought I was feeling good about this but now I don't even myself know

what the answer is our professional baseball players allowed to glue small Wings onto their caps we have two confident answers that are contradictory across two models that are very closely

related starting to worry us a little bit I hope but still it's impressive what's that you could check yes I have a few cases

and this is an interesting experiment for us to run for sure let me show you the responses I got a bit later the point though I guess if you've seen the movie Blade Runner this is starting

to feel like to figure out whether an agent we were interacting with was human or AI we would need to get very sophisticated sophisticated interview

techniques indeed the touring test long forgotten here now we're into the mode of trying to figure out uh exactly what kind of Agents we're interacting with by

having to be extremely clever about the kinds of things that we do with them now that's kind of anecdotal evidence but I think that the picture of progress

is also supported by what's happening in the field let me start this story with our benchmarks and the headline here is that our benchmarks the tasks the data

sets we use to probe our models are saturating faster than ever before and I'll articulate what I mean by saturate so we have a little framework along the

x-axis I have time stretching back into like the 1990s and along the y-axis I have a normalized measure of distance from what we call Human Performance

that's the red line set at zero each one of these benchmarks has in its own particular way set a so-called estimate of human performance I think we should be cynical about that but nonetheless

it'll this will be a kind of marker of progress for us First Data set mnist this is like digit recognition famous task in AI it was

launched in the 1990s and it took about 20 years for us to see a system that surpassed human performance in this very loose sense the switchboard Corpus this is going

from speech to text it's a very similar story launched in the 90s and it took about 20 years for us to see a superhuman system

imagenet this was launched I believe in 2009 and it took less than 10 years for us to see a system that surpassed that red line and now progress is going to pick up

really fast Squad 1.1 the Stanford question answering data set was launched in 2016 and it took about three years for it to be saturated in this sense

Squad 2.2.0 was the team's attempt to pose an even harder problem one where they were unanswerable questions but it took even less time for systems to get

past that red line then we get the glue Benchmark this is a famous Benchmark in natural language understanding a multi-task benchmark when this was launched a lot of us

thought that glue would be too difficult for present-day systems it looked like this might be a challenge that would stand for a very long time but it took like less than a year for systems to

pass Human Performance the response was super glue but it was saturated if anything even more quickly now we can be as cynical as we want about this notion of human performance

and I think we should dwell on whether or not it's fair to call it that but even setting that aside this looks like undeniably a story of progress the

systems that we had in 2012 would not even have been able to enter the glue Benchmark to say nothing of achieving scores like this so something meaningful has happened now you might think by the

standards of AI these data sets are kind of old here's a post from Jason way where he evaluated our latest and greatest large language models on a bunch of mostly new tasks that were

actually designed to stress test this new class of very large language models and Jason's observation is that we see emergent abilities across more than a hundred tasks for these models

especially for our largest models the point though is that we again thought these tasks would stand for a very long time and what we're seeing instead is that one by one systems are certainly

getting traction and in some cases performing at the standard we had set for humans again an incredible story of progress there so I hope that is energizing maybe a

little intimidating but I hope fundamentally energizing for you all uh the next question that I want to ask for you is just what is going on what is

driving all of this sudden progress let's get a feel for that and that'll kind of serve as the foundation for the course itself before I do that though are there questions or comments

things I could resolve or things I left out about the current moment write it on board it did very well we should we should reflect though maybe as a group about what it means to do

very well my my question for you when you say it did well what is the Major League Baseball rule about players gluing things onto their caps

you found the actual rule no Bard found that rule and gave me that number okay yes that is going to be the question for us I can get which is right for

hallucination well I'm going to show you the open AI models will offer me links but the links go nowhere [Music] what you're pointing out I think is an

increasing societal problem these models are offering us what looks like evidence but a lot of the evidence is just fabricated and this is worse than offering no evidence at all what I

really need is someone who knows Major League Baseball to tell me what is the rule about players and their caps I want it from an expert human not an expert language model

be careful how you Google though I guess that's the lesson of 2023 all right what's going on let's start to make some progress on this again first a

little bit of historical context I've got a timeline going back to the 1960s along the x-axis this is more or less the start of the field itself and in

that early era essentially all of the approaches were based in symbolic algorithms like the chat 81 that I showed you in fact you know that was kind of pioneered here at Stanford by

people who were pioneering the very field of AI and that Paradigm of essentially programming these systems lasted well into the 1980s

in the 90s early 2000s we get the statistical Revolution throughout artificial intelligence and then in turn in natural language processing and the big change there is that instead of

programming systems with all these rules we're going to design machine learning systems that are going to try to learn from data under the hood there was still a lot of programming involved because we would write a lot of feature functions

that were little programs that would help us detect things about data and we would hope that our machine Learning Systems could learn from the output of those feature functions but in the end

this was the rise of the fully data-driven learning systems and we just hope that some process of optimization leads us to new capabilities

the next big phase of this was the Deep learning Revolution this happened starting around 2009-2010 again Stanford was at the at the Forefront of this to be sure it felt

like a big change at the time but in retrospect this is kind of not so different from this mode here it's just that we now replace that simple model

with really big models really deep models that have a tremendous capacity to learn things from data we started also to see a shift even further away

from those feature functions from writing little programs and more toward a more mode where we would just hope that the data and the optimization

process could do all the work for us then the next thing big thing that happened which I could could take us I suppose until about 2018 would be this mode where we have a lot of pre-trained

parameters these are pictures of maybe big language models or computer vision models or something and when we build systems we build on those pre-trained components and Stitch them together with

these tasks specific parameters and we hope that when they're all combined and we do some learning on some tasks specific data we have something that's benefiting from all these pre-trained components

and then the mode that we seem to be in now that I want us to reflect critically on is this mode where we're going to replace everything with maybe one

ginormous language model of some kind and hope that that thing that enormous black box will do all the work for us I think we should think critically about whether that's really the path forward

but it certainly feels like the Zeitgeist to be sure question yeah um if you think it's worth it could you go back to the last slide um and maybe

explain a little bit a more grounded example of what that all means I couldn't quite follow let's do that later the point for now though is really this shift from here where we're mostly

learning from scratch from our for our task here we've got things like Bert in the mix we've got you know pre-trained components models that we hope begin in

a state that gives us a leg up on the problem we're trying to solve that's the big thing that happened and you get this emphasis on people releasing model parameters you know in this earlier

phase like here there was no talk of releasing model parameters because mostly the models people trained were just good for the tasks that they had set as we move into this era and then

certainly this one these things are meant to be like general purpose language capabilities or maybe general purpose computer vision capabilities that we stitch together into a system

that can do more than any previous system could do right so then we have this big thing here so that's the feeling now behind all of

this certainly beginning in this Final Phase here is the Transformer architecture just may take the temperature of the room how many people have encountered the Transformer before

right yeah it's sort of unavoidable if you're doing this research here's a diagram of it but I'm not going to go through this diagram now because starting on Wednesday we are going to

have an entire lecture essentially devoted to unpacking this thing and understanding it all I can say for you now is that I expect you to go on the following Journey which all of us go on

how on Earth does the Transformer work it looks very very complicated I I hope can get you to the point where you feel oh this is actually pretty simple components that have come been combined

in a pretty straightforward way that's your second step on the journey the one the true Enlightenment comes from wait a second why does this work at all and then you're with the entire field

trying to understand why these simple things were brought together in this way have proved so powerful the other major thing that happened

which is kind of latent going all the way back to the start of link of AI especially as it relates to Linguistics is this notion of self-supervision of distributional learning because this is

going to unlock the door to us just learning from the world in the most General sense in self-supervision your model's only goal is to learn from co-occurrence

patterns in the sequences that it's trained on the sequences can be language but they could be language plus sensor readings computer code maybe even images that you embed in this space just

symbols and the model's only goal is to learn from the distributional patterns that they contain or for many of these models to assign high probability to the attested sequences in whatever data that you pour

in for this kind of learning we don't need to do any labeling all we need to do is have lots and lots of symbol streams

and then when we generate from these models we're sampling from them and that's what we all think of when we think of prompting and getting a response back but the underlying mechanism is at least in part this

notion of self-supervision and I'll emphasize again because I think this is really important for why these models are so powerful the symbols do not need to be just language they can include

lots of other things that might help a model piece together a full picture of the world we live in and also the connections between language and those pieces of the world just from this

distributional learning the result of this proving so powerful is the Advent of large-scale pre-training because now we're not held

back anymore by the need for labeled data all we need is lots of data in unstructured format this really begins in the era of static word representations like word to vac and

glove and in fact those teams and I would say especially the glove team they were really Visionary in the sense that they not only released a paper

and code but pre-trained parameters this was really brand new for the field this idea that you would Empower people with model artifacts and people started using

them as the inputs to recur recurrent neural networks and other things and you started to see pre-training as an important component to doing really well at hard things

there were some predecessors that I'll talk about next time but the really big moment for contextual representations is the Elmo model this is the paper deep contextualized word representations I

can remember being at the North American ACL meeting in New Orleans in 2018 at the best paper session they had not announced which of the best papers was

going to win the outstanding paper award but we all knew it was going to be the Elmo paper because the gains that they had reported from fine-tuning their Elmo

parameters on hard tasks for the field were just mind-blowing the sort of thing that you really only see once in a kind of generation of This research or so we

thought because the next year Bert came out same thing I think same best paper award thing the paper already had had huge impact by the time it was

even published and they too released their model parameters Elmo is not Transformer based Bert is the first of the sequence of things that's based in the Transformer and again lifting all

boats even above where Elmo had brought us then we get GPT this is the first GPT paper and then fast forward a little bit

we get gpt3 and that was pre-training at a scale that was previously kind of unimaginable because this we're now we're talking about you know for the

Bert model 100 million parameters and for gpt3 well north of 100 billion different order of magnitude and what we

started to see is emergent capabilities that model size thing is important again this is a sort of feeling of progress and maybe also despair I think I can

lift your spirits a little bit but we should think about model size so I have years along the x-axis again and I have model size going from 100 million to 1 trillion here on a

logarithmic scale so 2018 GPT that's like 100 million Bert I think it's 300 million for the large

one okay gbt2 even larger Megatron 8.3 billion I remember when this came out I probably laughed maybe I thought it was a joke I certainly thought it was some kind of typo because I couldn't imagine

that it was actually billion like with a B there um but now that's you know we take that for granted Megatron 11 billion this is

20 21 or so then we get gpt3 reportedly at 175 billion parameters and then we get this thing where it seems like we're doing typos again Megatron Turing nlg

was like 500 and then Palm is 540 billion parameters and I guess there are rumors that we have gone upward all the way to a trillion right there's an

undeniable Trend here I think there is something to this trend um but we should reflect on it a little bit one thing I want to say is there's a

noteworthy pattern of very few entities have participated in this very large in this race for very large models we've got like Google Nvidia meta and open AI

right and that was actually a real cause for concern I remember being at a workshop between Stanford and open AI where the number one source of

consternation was really that only open AI at that point had trained these really large models and after that predictably these other large tech companies kind of caught up

but it was still for a while looking like a story of real centralization of power that might still be happening but I think there's a reason to be optimistic so here at Stanford the helm Group which

is part of the center for research on Foundation models LED this incredibly ambitious project of evaluating lots of language models and one thing that emerges from that is that we have a more

healthy ecosystem now so we have these like loose collectives big science and a Luther are both kind of fully open source groups of researchers we've got well one academic institution

represented this could be a little bit embarrassing for Stanford maybe we'll correct that and then maybe the more important thing is that we have lots of startups represented so these are well funded but relatively small outfits that

are producing outstanding language models and so the result I think we're going to see much more of this and then we'll worry less about centralization of power there's plenty of other things to

worry about so we shouldn't get sanguine about this but this particular point I think is being alleviated by current trends and there's another aspect of this too which is I have the scary ryzen

model size but what is happening right now as we speak in a very quick way is we're seeing a push towards smaller models and in particular we're seeing that models that are in the range of

like 10 billion parameters can be highly performant right so we have the flan models we have llama and then here at Stanford they release the alpaca thing

and then uh data bricks released Hello Dolly model these are all models that are like eight to ten billion parameters which I know this sounds funny because I laughed a few years ago when the

Megatron model had 8.3 billion and now what I'm saying to you is that this is relatively small but so it goes and the point is that a 10 billion parameter model is one that could be run on

regular old commercial Hardware whereas these monsters up here really you have lots of pressures towards centralization of power there because almost no one can work with them but anyone essentially

can work with alpaca and it won't be long before we've got the ability to kind of work with it on small devices and things like that and that too is really going to open the door to lots of

innovation I think that will bring some good and I think it will bring some bad but it is certainly a meaningful change from this scary Trend that we were seeing until four months ago

as a result of these models being so powerful people started to realize that you can get a lot of mileage out of them simply by prompting them when you prompt

one of these very large models you put it in a temporary state by inputting some text and then you generate a sample from the model using some technique and you see what comes out right so if you type into one of these models better

late than it's going to probably spit out never if you put in every day I eat breakfast lunch and it will probably say dinner and you might have an intuition that the

reasons the causes for that are kind of different the first one is a sort of idiom so that it could just learn from co-occurrence patterns and text transparently for the second one we kind

of interpret it as humans as reflecting something about routines but you should remind yourself that the mechanism is the same as in the first case this was just a bunch of

co-occurrence patterns a lot of people describe their routines in text and the model picked up on that and carry that thought forward as you think about things like the president of the US is

when it fills that in with Biden or whoever it might look like it is offering us factual knowledge and maybe in some sense it is but it's the same mechanism

as for those first two examples it is just learning from the fact that a lot of people have expressed a lot of texts that look like the president of the US is Joe Biden and it is repeating that back to us

and so definitely if you ask a model something like the key to happiness is you should remember that this is just the aggregate of a lot of data that it was trained on it has no particular

wisdom to offer you necessarily beyond what was encoded latently in that giant sea of mostly unaudited unstructured

text Yeah question yes yes I guess you know it'd be kind of hard to get something like this but if we had like a corpus of just like you know all the like language is right but

literally all of the facts were wrong like we just imagined like a very factually Ingrid course like like I I guess I'm getting at like how do we inject like truth into like

these horses uh it's a question that bears repeating how do we inject truth it's a question you all could think about what is truth of course but also

um what would that mean and how would we achieve it and even if we did back off to something like how would we ensure self-consistency for a model or you know at the level of a world view or a set of

facts even those questions which seem easier to pose are incredibly difficult questions in the current moment where our only mechanisms are basically that self-supervision thing that I described

and then a little bit of what all about what I'll talk about next but none of the structure that we used to have where we would have a database of knowledge and things like that that is posing

problems the prompting thing we take this a step forward right so the gpt3 paper remember that's that 175 billion parameter monster the eye-opening thing about that

is what we now call in context learning which was just the notion that for these very large very capable models you could input a bunch of text like here's a passage

and maybe an example of the kind of behavior that you wanted and then your actual question and the model would do a pretty good job at answering the question what you're doing here is with your

context passage and your demonstration pushing the model to be extractive to find an answer to its question in the context passage and then the observation

of this paper is that they do a pretty good job at following that same behavior for the actual Target question at the bottom here remember this is all just prompting putting the model in a

temporary State and seeing what comes out you don't change the model you just prompt it this in 2012 if you had asked me whether this was a viable path forward for a

class project I want to prompt an RNN or something I would have advised you as best I could to choose some other topic because I never would have guessed that this would work so the mind-blowing

thing about this paper and everything that's followed is that we might be nearing the point where we can design entire AI systems on the basis of this

simple in context learning mechanism transformatively different from anything that we saw before in fact let me just emphasize this a little bit it is worth dwelling on how

strange this is for those of you who have been in the field a little while just contrast what I described in context learning with the standard mode

of supervision let's imagine for a case here that we want to train a model to detect nervous anticipation and I have picked this because this is a very particular human

emotion and in the old mode we would need an entire dedicated model to this right we would collect a little data set of positive and negative instances of

nervous anticipation and we would train a supervised classifier on feature representations of these examples over here learning from this binary distinction

we would need custom data and a custom model for this particular task in all likelihood in this new mode few shot in context learning we essentially just prompt the

model hay model here's an example of nervous anticipation my palm started to sweat as the lotto numbers were read off hey model here's an example without

nervous anticipation and so forth and it learns from all those symbols that you put in in their co-occurrences something about nervous anticipation

on the left for this model here I've written out nervous anticipation but remember that has no special status I've structured the model around the binary distinction the one and the zero and

everything about the model is geared toward my learning goal on the right nervous anticipation is just more of the symbols that I've put into the model

and the eye-opening thing again about the gpt3 paper and what's followed is that models can learn be put in a temporary State and do well at tasks like this

now I talked about self-supervision before and I think that is a major component to the success of these models but it is increasingly clear that it is not the

only thing that is Driving Learning in the best models in these in this class the other thing that we should think about is what's called reinforcement

learning with human feedback this is a diagram from the chat GPT blog post there are a lot of details here but really two of them are important for us

for right now the first is that in a phase of training these models people are given inputs and asked themselves to produce good outputs for

those inputs so you might be asked to do a little Python program and you yourself as an annotator might write that Python program for example so that's highly skilled work that depends on a lot of

human intelligence and those examples those pairs are part of how the model is trained and that is so important because that takes us Way Beyond just learning

from co-occurrence patterns of symbols and text it is now back to a very familiar story from all of AI which is that it's not magic what is happening is

that a lot of human intelligence is is driving the behavior of these systems and that happens again at step two here so now the model produces different

outputs and humans come in and rank those outputs again expressing direct human preferences that take us well beyond self-supervision so we should remember we had that brief moment where

it looked like it was all unstructured unlabeled data and that was important to unlocking these capacities but now we are back at a very labor-intensive human

capacity here driving what looked like the really important behaviors for these models final step which I think actually intimately relates to that instruct tuning that I just described that's a

kind of way of summarizing this reinforcement learning with human feedback and this is what's called step-by-step or Chain of Thought reasoning now we're thinking about the prompts that we use for these models

so suppose we asked ourselves a question like can models reason about negation to give an example does the model know that if the customer doesn't have any auto loan sorry doesn't have any loans then

the customer doesn't have any auto loans it's a simple example it's the sort of reasoning that you might have to do if you're thinking about a car contract or something like that whether a rule has

been followed and it just involves negation our old friend from the start of the lecture now in the old school prompting style

all the way back in 2021 we would kind of naively just input is it true that if the customer doesn't have any loans then the customer doesn't have any auto loans into one of these models and we would

see what came back and here it says no this is not necessarily true a customer can have auto loans without having any other loans which is the reverse of the question that I asked again kind of

showing it doesn't deeply understand what we've put in here it just kind of does an act that looks like it did and that's worrisome but we're learning how to communicate with these very alien

creatures now we do what's called step-by-step prompting this is The Cutting Edge thing you would just tell the model that it was in some kind of logical or common sense reasoning exam That Matters to the model

then you could give some instructions and then you could give an example in your prompt of the kind of thing it was going to see and then finally you could prompt it with your premise and then your question

and the model would spit out something that looked really good here I won't bother going through the details but with that kind of prompt the model now

not only answers and reasons correctly but also offers a really nice explanation of its own reasoning the capacity was there it was latent and we didn't see it in the simple prompting

mode but the more sophisticated prompting mode elicited it and I think this is in large part the result of the fact that this model was instruct tuned and so people actually

taught it about how that markup is supposed to work and how it's supposed to think about prompts like this so the combination of all that human intelligence and the capacity of the

model led to this really interesting and much better Behavior that is a glimpse of the foundations of all of this I would say of course we're going to unpack all of that stuff as we

go through the quarter but I hope you're getting a sense for it are there questions I can answer about it things I could Circle back on yes the human brain

has about 100 billion neurons is my understanding uh and I'm not sure how many parameters or something are we approaching a point where these

machines can start emulating the human brain or is there something to the language in Spain or you know instincts of all kinds that maybe take into the human rights

oh it's nothing but big questions today right so so the question is kind of like what is the relationship between the models we're talking about and the human brain and you you raise that in terms of

the size and I guess the upshot of your description was that these models remain smaller than the human brain yeah I think that's reasonable um It's Tricky though on the one hand they obviously have superhuman

capabilities um on the other hand they fall down in ways that humans don't it's very interesting to ask why that difference exists um and maybe that would tell us

something about the limitations of learning from scratch versus being initialized by Evolution the way all of us were I don't know but I would say that

underlying your whole line of questioning is the question can we use these models to eliminate questions of Neuroscience and cognitive science and I

think we should be careful but that the answer is absolutely yes and in fact the increased ability of these models to learn from data has been really

Illuminating about certain kind of recalcitrant questions from cognitive science in particular you have to be careful because they're so different from us these models on the other hand I

think they are helping us understand how to differentiate different theories of cognition and ultimately I think they will help us understand cognition itself yeah yeah and I would of course welcome

projects that were focused on those cognitive questions in here this is a wonderful space in which to explore this kind of more speculative angle

connecting AI to the cognitive Sciences other questions comments yes in the back I'm curious to understand whether I mean harshly following up on the brain thing

just to use a metaphor of our brain not being just one huge lump of neurons but being separated into different areas and then also thinking about the previous phase that you talked about about

breaking up the models and potentially having a model in the front that decides which domain or question falls into and then having different soft models and I'm wondering whether

that's arising whether I want to touch on an architecture like that um because it just seems natural to me because prompting a huge model is just

very expensive computationally it feels like combining big models and logic trees could be a cool approach I love it yeah like one quick summary of

what you said would relate directly to your question the modularity of mind is an important old question about human cognition to what extent are our

abilities modularized in the mind brain uh with these current models which have a capacity to do lots of different things if they have the right pre-training in

the right structure we could ask does modularity emerge naturally or do they learn non-modular Solutions both of those seem like they could be indirect evidence for how people work again we

have to be careful because these models are so different from us but as a kind of existence proof for example that modularity was emergent from otherwise unstructured learning that would be

certainly eye-opening right I have no idea yeah I don't know whether the results for that their results just kind of a follow-up and on that

and all these big models are how could we interact with the model of such a way that helps us learn if there is modulars because we literally can only attract

so how do we go about studying that right so the question is you know the the closed off nature of a lot of these models has been a problem we can access the open AI models but only through an

API we don't get to look at their internal representations and that has been a blocker but I mentioned the rise of these 10 billion parameter models as being performant and interesting and

those are models that with the right Hardware you can dissect a little bit and I think that's just going to get better and better and so we'll be able to you know peer inside them in ways that we haven't been able to until

recently yeah and in fact like we're going to talk a lot about explainability that's a major unit of this course and I think it's an increasingly important area of the whole

field that we have techniques for understanding these models so that we know how they're going to behave when we deploy them and it would be wonderfully exciting if you all wanted to try to scale the methods we talk about to a

model that was as big as 8 or 10 billion parameters ambitious just to do that but then maybe a meaningful step forward yeah um I have a question back to like this

baseball cap comp that we were discussing was it like there is a little bit of ambiguity for like human interpretation like for example in the honor code in the fundamental standard like it's intentionally ambiguous so that it's

context dependent um and so like the idea that there's like this inherent underlying value system that like affords whatever the rules that are written out are and so

that's like the primary form of evaluation and so I guess like how does that play into then how these language models are understanding like is there some form of encoded or understanding

understood deeper value system that's encoded into them you could certainly ask I mean the essence of your question is could we with analysis techniques say uh find out

that a model had a particular belief system that was guiding Its Behavior I think we can ask that question now it sounds fantastically difficult but maybe piecemeal we could get make some

progress on it for sure yeah I want to return to the MLB one though because well as you'll see and as I think we already saw these models purport to offer evidence from a rule book

and that's where I feel stuck the answer and some other stuff in the class discussion wonderful thank you yes can we just hook up these models to

a large database information web oh well kind of Yes actually this is the sort of solution that I want to advocate for I'm going to do this in a minute yeah here let's uh so we'll do this overview

I want to give you a feel for how the course will work and then uh dive into some of our major themes so high level overview we've got these topics contextual representations Transformers and stuff multi-domain sentiment

analysis that will be the topic of the first homework and it's going to build on the first unit there retrieval augmented in context learning this is where we might hook up to a database and get some guarantees about how these

models will behave compositional generalization in case you were worried that all the tasks were solved I'm going to confront you with a task a seemingly simple task about semantic interpretation that you will

well I think it will not be solved I mean those could be famous last words because who knows what you all are capable of but it's a very hard task that we will pose we'll talk about benchmarking and adversarial training and testing

increasingly important topics as we move into this mode where everyone is interacting with these large language models and feeling impressed by their behavior we need to take a step back and rigorously assess whether they actually

are behaving in good ways or whether we're just biased toward remembering the good things and forgetting the bad ones we'll do model introspection that's the explainability stuff that I mentioned and finally methods and metrics and as

you can see for the like five six and seven that's going to be in the phase of the course where you're you're focused on Final projects and I'm hoping that that gives you tools to write really

rich final papers that have great analysis in them and really excellent assessments and then for the work that you'll do we're going to have three assignments and each one of the assignments is

paired with what we call a bake off which is an informal competition around data and modeling essentially the homework problems ask you to set up some baseline systems and get a feel for a

problem and then you write your own original system and you enter that into the bake off and we have a Leaderboard on grade scope and the team is going to look at all your submissions and give

out some prizes for top performing systems but also systems that are really creative or interesting or ambitious or something like that and that has always been a lot of fun and also really

Illuminating because it's like crowdsourcing a whole lot of different approaches to a problem and then as a group we can reflect on what worked and what didn't and look at the really

ambitious things that you all try so that's my favorite part we have three offline quizzes and this is just as a way to make sure you have incentives to really immerse yourself in the course

material and those are done on canvas there's actually a fourth quiz which I'll talk a little bit about probably next time that is just making sure you understand the

course policies that's quiz zero you can take it as many times as you want but the idea is that that you will have some incentive to learn about policies like due dates and so forth

and then the real action is in the final project and that will have a lit review phase an experiment protocol and a final paper those three components you'll probably do those in teams and

throughout all of that work you'll be mentored by someone from the teaching team and as I said before we have this incredibly expert teaching team lots of varied expertise a lot of experience in

the field and so we hope to align you with the person the with someone who's really aligned with your project goals and then I think you can go really really far

forward to because I don't stand for kids get obsessed about this stuff on the final project is this more of an academic paper or a rather about

building working code and showing of state of the art great question for the first one the bake offs yes it is easy to get obsessed with your Bake Off entry I would say that if you

get obsessed and you do really well just make that into your final project all three of them all three of them are really important problems they are not idle work I mean one of them is on

retrieval augmented in context learning which is one of my core research focuses right now so is compositional generalization if you do something really interesting for a bake off make it your final paper and then go on to

publish it for the second part of your question I would say that the core goal is to get you to produce something that could be a research contribution in the field and we have lots of success

stories I've got Links at the website to people who have gone on to publish their final paper as an NLP paper I I'm careful the way I say that they didn't literally publish the final paper

because in 10 weeks almost no one can produce a publishable paper it's just not enough time but you could form the basis for then working a little bit more or a lot more and then getting a really

outstanding publication out of it and I would say that that's the default goal the nature of the contribution though is highly varied we have one requirement which is that the final paper have some quantitative evaluation in it but there

are a lot of ways to satisfy that requirement and then you could be serving many different questions in the field for some expansive notion of the field as well

background materials so I should say that officially we are presupposing cs224n or cs224s as prerequisites for the course and what that means is that

I'm going to skip a lot of the fundamentals that we have covered in past years if you need a refresher check out the background page of the course site it covers fundamentals of scientific

computing static Vector representations like word to VEC and glove and supervised learning and I'm hoping that that's enough of a refresher if you look at that material

and find that it too is kind of Beyond where you're at right now then contact us on the teaching team and we can think about how to manage that but officially

this is a course that presupposes cs224n then the core goals this kind of relates to that previous question hands-on experience with a wide range of problems mentorship from the teaching team to

guide you through projects and assignments and then really the central goal here is to make you the best that is most insightful most responsible most

flexible nlu researcher and practitioner that you can be for whatever you decide to do next and we're assuming that you have lots of diverse goals that somehow connect with nlu

all right let's do some course themes unless there are questions I have a whole final section of this slideshow that's about the course materials and requirements and stuff

might save that for next time and you can check it out at the website and you'll be forced to engage with it for quiz zero um I thought instead I would dive back into the content part of this unless

there are questions or comments all right first course theme Transformer based pre-training so starting with the Transformer we want to talk about Core Concepts and goals

give you a sense for what these models are like why they work what they're supposed to do all of that stuff we'll talk about a bunch of different architectures there are dozens and

dozens of them but I hope that I have picked enough of them with the right selection of them to give you a feel for how people are thinking about these models and the kind of Innovations they brought in that have led to real

meaningful advancement just at the level of architectures we'll also talk about positional encoding which I think maybe a lot of us have been surprised to see just how important that is as a differentiator for different approaches

in this space we'll talk about distillation taking really large models and making them smaller it's an important goal for lots of reasons and an exciting area of

research and then as I mentioned it's going to do a little lecture for us on diffusion objectives for these models and then it's going to talk about practical pre-training and fine-tuning

I'm going to enlist the entire teaching team to do guest lectures and these are the two that I've lined up so far and that will kind of culminate or be aligned with this first homework and

Bake Off which has a multi-domain sentiment I'm going to give you a bunch of different sentiment data sets and you're going to have to design one system that can kind of succeed on all of them and then for the bake off we

have an unlabeled data set for you we have the labels but you won't and that has data that's like what you developed on and then some mystery examples that you will not really be able to

anticipate and we're going to see how well you do at handling all of these different domains with one system and this is by way of kind of again a

refresher On Core Concepts and supervised learning and really getting you to think about Transformers although we're not going to constrain the kind of solution that you offer for your original system

our second major theme will be retrieval augmented in context learning a topic that I would not even have dreamed of five years ago and seem kind of

infeasible three years ago and that we first did two years one year ago oh goodness I think this is only the second time but I had to redo it entirely because things have happened changed so

much um here's the idea we have two characters so far in our kind of emerging narrative for nlu on the one hand we have this approach that I'm

going to call llms for everything large language models for everything you input some kind of question here I've chosen a very complicated question which MVP of a game read Flaherty

umpired was elected to the Baseball Hall of Fame and hats off to you if you know that the answer is Sandy Koufax the llms for everything approach is that

you just type that question in and the model gives you an answer and hopefully you're happy with the answer the other character that I'm going to introduce here is what I'm going to call

retrieval augmented so I have the same question at the top here except now this is going to proceed differently the first thing that we will do is take some large language model and encode that

query into some numerical representation that's sort of familiar the new piece is that we're going to also have a

knowledge store which you could think of as an old-fashioned web Index right just a knowledge store of documents with the modern twist that now all of the

documents are also represented by large language models but fundamentally this is an index of the sort that drives all web search right now we can score documents with respect to

queries on the basis of these numerical representations and if we want to we can reproduce the classic search experience here I've got a ranked list of documents

that came back from my query just like when you do Google as of the last time I Googled but in this mode we can continue right we could have another language model

slurp up those retrieved documents and synthesize them into an answer and so here at the bottom I've got it's kind of small but it's the same answer over here although notably this answer is now

decorated with links that would allow you the user to track back to what documents actually provided that evidence whereas on the left who knows where that information came

from and that's kind of what we were already grappling with this is an important societal need because this is taking over web search what are our goals for this kind of

model here so first we want synthesis fluency right we want to be able to take information from multiple documents and synthesize it down into a single answer and I think both of them approaches that

I just showed you are going to do really well on that we also need these models to be efficient to be updatable because the world is changing all the time

we need it to track Providence and maybe invoke something like factuality but certainly Providence we need to know where the information came from and we need some Safety and Security we need to know that the model won't produce

private information and we might need to restrict access to parts of the model's knowledge to different groups like different customers or different people with different Privileges and so forth that's what we're going to need if we're

really going to deploy these models out into the world as I said I think both of the approaches that I sketch do well on the synthesis part because they both use a language model and those are really good they all

have the Gift of Gab so to speak what about efficiency right on the llm for any for everything approach we had this undeniable rise in model size and I

pointed out models like alpaca that are smaller but I strongly suspect that if we are going to continue to ask these models to be both a knowledge store and a language

capability we're going to be dealing with these really large models the hope of the retrieval augmented approach is that we could get by with the smaller models and the reason we

could do that is that we're going to factor out the knowledge store into that index and the language capability which is going to be the language model the only thing we're going to be asking the

language model is to be good at that kind of in-context learning it doesn't need to also store a full model of the world and I think that means that these

models could be smaller so overall a big gain in efficiency if we go retrieval augmented people will make progress but I think it's going to be tense

what about updateability again this is a problem that people are working on very concertedly for the llms for everything approach but these models persist in giving outdated answers to questions

and one pattern you see is that there's a lot of progress where you could like edit a model so that it gives the correct answer to who is the president of the US but then you ask it about

something related to the family of the president and you re it reveals that it has outdated information stored in its parameters and that's because all of this information is interconnected and

we don't at the present moment know how to reliably do that kind of systematic editing okay on the retrieval augmented approach

we just re-index our data right if the world changes we assume that the knowledge store changed like somebody updated a Wikipedia page so we represent all the documents again or at least just

the ones that changed and now we have a lot of guarantees that as that propagates forward into the retrieved results which are consumed by the language model it will reflect the changes we made to the underlying

database in exactly the same way that a web search index is updated now right one forward pass of the large language model compared to maybe

training from scratch over here on new data to get an absolute guarantee that the change will propagate what about Providence okay we have seen

this already this problem here llms for everything I asked um gpt3 The DaVinci 3 Model my question are professional baseball players allowed to

glue small Wings onto their caps but I kind of cut it off but at the top there I said provide me some links to the evidence and it dutifully provided the links but

none of the links are real if you copy them out and follow them they all go to 404 pages and I think that this is worse than providing no Links at all because I'm

attuned as a human in the current moment to see links and think they're probably evidence and I don't follow all the links uh and here you might look and say

oh yeah I see it found the relevant MLB pages and that's it right over here the kind of the point of this is that we are first doing a search phase where we're

actually linked back to documents and then we just need to solve the interesting non-trivial question of how to link those documents into the synthesized answer but all of the information we need is right there on

the screen for us and so this feels like a relatively tractable problem compared to what we were faced with on the left I will say I've been in just amazed at the

rollout especially of of the Bing search engine which now incorporates open AI models at some level because it is clear that it is doing web search right because it's got information that comes

from documents that only appeared on the web days before your query but what it's doing with that information seems completely chaotic to me so that it's kind of just getting mushed

in with whatever else the model is doing and you get this unpredictable combination of things that are grounded in documents and things that are completely fabricated and again I

maintain this is worse than just giving an answer with no evidence attached to it why these companies are not simply doing the retrieval augmented thing but I'm sure they are going to wise up and maybe

your research could help them wise up a little bit about this finally Safety and Security this is relatively straightforward on the llms for everything approach we have a pressing problem

privacy challenges we know that those models can memorize long strings in their training data and that could include some very particular information about one of us and that should be worrying us

we have no known way with a language model to compartmentalize llm capabilities and say like you can see this kind of result and you cannot and similarly we have no known way to

restrict access to part of an llm's capabilities they just produce Things based on their prompts and you could try to have some prompt tuning that would tell them for this kind of person or setting do this

and not that but nobody could guarantee that that would succeed whereas for the retrieval augmented approach again we're thinking about accessing

information from an index and access restrictions on an index is an old Problem by now again I don't want to say solved but something that a lot of

people have tackled for decades now and so we can offer something like guarantees just from the fact that we have a separated knowledge store again my smiley face you can see where

my feelings are for the llms for everything approach you know people are working on these problems and it's very exciting and if you want a challenge take you know take up one of these challenges here but over here on the

retrieval augmented side I think we have lots of reasons to think it's not that they're completely solved it's just that we can see the path to solving them and this feels very urgent to me because of

how suddenly this kind of technology is being deployed in a very user-facing way for one of the core things we do in society which is web search so it's it's an urgent thing that we get

good at this final things I want to say about this so until recently the way you would do even the retrieval augmented thing would be that you would have your index and then

you might train a custom purpose model to do the question answering part and it could extract things from the text that you produced or maybe even generate some new things from the text that you produced and that's kind of the mode that I

mentioned before where you'd have like some language models maybe a few of them and you'd have an index and you would stitch them together into a question answering system that you would probably train on question answering data and you

would hope that this whole big monster may be fine-tuned on squad or natural questions or one of those data sets gave you a general purpose question answering

capability that's the present but I think it might actually be the recent past and in fact the way that you all will probably work

when we do this unit and certainly for the homework is that we will just have Frozen components this starts from the observation that the retriever model is

really just a model that takes in text and produces text with scores and a language model is also a device for taking in text and producing text with scores

these are when these are frozen components you can think of them as just black box devices that do this input output thing and then you get into the Intriguing mode of asking well what if

we had them just talk to each other and that is what you will do for the homework and Bake Off You will have Frozen Retriever and a Frozen large language model and you will get them to

work together to solve a very difficult open domain question answering problem and that's kind of pushing on us into a new mode for even thinking about how we

design AI systems where it's not so much about fine-tuning it's much more about getting them to communicate with each other effectively to design a system

from Frozen components again unanticipated at least by me as of a few years ago and now an exciting New Direction so just to wrap out I think what I'll do

since we're near the end of the of class here I'll just finish up this one unit and then we'll use some of our time next time to introduce a few other of these course themes and that'll set us up well

for diving into Transformers final piece here just to inspire you few shot open QA is kind of the task that you will tackle for homework too and here's how you could think about this imagine that

the question has come in what is the course to take the most standard thing we could do is just prompt the language model with that question what what is the course to take down here and see what answer it gave

back right but the retrieval augmented Insight is that we might also retrieve some kind of passage from a knowledge store here I have a very short passage the course to take is natural language

understanding and that could be done with a retrieval mechanism but why stop there it might help the model as we saw going back to the gpt3

paper to have some examples of the kind of behavior that I'm hoping to get from the model and so here I have retrieved from some data set question answer pairs that will kind of give it a sense for

what I want it to do in the end but again why stop there we could also pick questions that were based very closely on the question that we posed

that would be like K nearest neighbors approach where we use our retrieval mechanism to find similar questions to the one that we care about I could also add in some context

passages and I could do that by retrieval so now we've used the retrieval model twice potentially wants to get good demonstrations and wants to provide context for each one of them

but I could also use my retrieval mechanism with the questions and answers from the demonstration to get even richer connections between my demonstrations and the passages I could

even use a language model to rewrite aspects of those demonstrations to put them in a format that might help me with the final question that I want to pose

so now I have an interwoven use of the retrieval mechanism and the large language model to build up this prompt right down at the retrieval thing I could do

the same thing and then when you think about the model generation again we could just take the top response from the model but we can do very sophisticated things on up to this full

retrieval augmented generation model which essentially marginalizes out the evidence passage and gives us a really powerful look at a good answer conditional on that very complicated

prompt that we constructed I think what you're seeing on the left here is that we are going to move from an era where we just type in prompts

into these models and hope for the best into an era where prompt construction is a kind of new programming mode where you're writing down computer code could

be python code that is doing traditional Computing things but also drawing on very powerful pre-trained components to assemble this kind of instruction kit

for your large language model to do whatever task you have set for it and so instead of Designing these AI systems with all that fine tuning I described

before we might actually be moving back into a mode that's like that symbolic mode from the 80s where you type in a computer program it's just that now the

program that you type in is connected to these very powerful modern AI components and we're seeing right now that that is opening doors to all kinds of new

capabilities for these systems and this first homework and Bake Off is going to give you a glimpse of that and you're going to use a programming model we've developed called demonstrate search

predict that I hope will give you a glimpse of just how powerful this can be all right we are out of time right 420.

um so next time I'll show you a few more units from the course and then we'll dive into Transformers