AI prompt engineering: A deep dive

- Basically, this entire roundtable session here

is just gonna be focused mainly on prompt engineering.

A variety of perspectives at this table around prompting

from a research side, from a consumer side,

and from the enterprise side.

And I want to just get the whole wide range of opinions

because there's a lot of them.

And just open it up to discussion

and explore what prompt engineering really is

and what it's all about.

And yeah, we'll just take it from there.

So maybe we can go around the horn with intros.

I can kick it off. I'm Alex.

I lead Developer Relations here at Anthropic.

Before that,

I was technically a prompt engineer at Anthropic.

I worked on our prompt engineering team,

and did a variety of roles spanning

from a solutions architect type of thing,

to working on the research side.

So with that, maybe I can hand it over to David.

- Heck, yeah. My name's David Hershey.

I work with customers mostly at Anthropic

on a bunch of stuff technical,

I help people with finetuning,

but also just a lot of the generic things

that make it hard to adopt language models of prompting.

And just like how to build systems with language models,

but spend most of my time working with customers.

- Cool. I'm Amanda Askell.

I lead one of the Finetuning teams at Anthropic,

where I guess I try to make Claude be honest and kind.

Yeah.

- My name is Zack Witten.

I'm a Prompt Engineer at Anthropic.

Alex and I always argue about who the first one was.

He says it's him, I say it's me.

- Contested. - Yeah.

I used to work a lot with individual customers,

kind of the same way David does now.

And then as we brought more solutions architects

to the team, I started working on things

that are meant to raise the overall levels

of ambient prompting in society,

I guess, like the prompt generator

and the various educational materials that people use.

- Nice, cool. Well, thanks guys for all coming here.

I'm gonna start with a very broad question

just so we have a frame

going into the rest of our conversations here.

What is prompt engineering? Why is it engineering?

What's prompt, really?

If anyone wants to kick that off,

give your own perspective on it,

feel free to take the rein here.

- I feel like we have a prompt engineer.

It's his job.

- We're all prompt engineers in our own form.

- But one of us has a job.

- Yeah. Zack, maybe since it's in your title.

- One of us has a job, but the other three don't have jobs.

- I guess I feel like prompt engineering

is trying to get the model to do things,

trying to bring the most out of the model.

Trying to work with the model to get things done

that you wouldn't have been able to do otherwise.

So a lot of it is just clear communicating.

I think at heart,

talking to a model is a lot like talking to a person.

And getting in there

and understanding the psychology of the model,

which Amanda is the world's most expert person in the world.

- Well, I'm gonna keep going on you.

Why is engineering in the name?

- Yeah.

I think the engineering part comes from the trial and error.

- Okay.

- So one really nice thing about talking to a model

that's not like talking to a person,

is you have this restart button.

This giant go back to square zero

where you just start from the beginning.

And what that gives you the ability to do

that you don't have, is a truly start from scratch

and try out different things in an independent way,

so that you don't have interference from one to the other.

And once you have that ability to experiment

and to design different things,

that's where the engineering part has the potential

to come in.

- Okay.

So what you're saying is as you're writing these prompts,

you're typing in a message to Claude or in the API

or whatever it is.

Being able to go back and forth with the model

and to iterate on this message,

and revert back to the clean slate every time,

that process is the engineering part.

This whole thing is prompt engineering all in one.

- There's another aspect of it too,

which is integrating the prompts

within your system as a whole.

And David has done a ton of work with customers integrating.

A lot of times it's not just as simple

as you write one prompt and you give it to the model

and you're done.

In fact, it's anything but. It's like way more complicated.

- Yeah.

I think of prompts as the way

that you program models a little bit,

that makes it too complicated.

'Cause I think Zack is generally right

that it's just talking clearly is the most important thing.

But if you think about it a little bit

as programming a model, you have to think about

where data comes from, what data you have access to.

So if you're doing RAG or something,

what can I actually use and do and pass to a model?

You have to think about trade-offs in latency

and how much data you're providing and things like that.

There's enough systems thinking

that goes into how you actually build around a model.

I think a lot of that's also the core

of why it maybe deserves its own carve-out as a thing

to reason about separately from just a software engineer

or a PM or something like that.

It's kind of its own domain

of how to reason about these models.

- Is a prompt in this sense then natural language code?

Is it a higher level of abstraction

or is it a separate thing?

- I think trying to get too abstract with a prompt is a way

to overcomplicate a thing, because I think,

we're gonna get into it, but more often than not,

the thing you wanna do

is just write a very clear description of a task,

not try to build crazy abstractions or anything like that.

But that said, you are compiling the set of instructions

and things like that into outcomes a lot of times.

So precision and a lot the things

you think about with programming about version control

and managing what it looked like

back then when you had this experiment.

And tracking your experiment and stuff like that,

that's all just equally important to code.

- Yeah.

- So it's weird to be in this paradigm where written text,

like a nice essay that you wrote is something

that's looked like the same thing as code.

But it is true that now we write essays

and treat them code, and I think that's actually correct.

- Yeah. Okay, interesting.

So maybe piggybacking off of that,

we've loosely defined what prompt engineering is.

So what makes a good prompt engineer?

Maybe, Amanda, I'll go to you for this,

since you're trying to hire prompt engineers

more so in a research setting.

What does that look like?

What are you looking for in that type of person?

- Yeah, good question.

I think it's a mix of like Zack said, clear communication,

so the ability to just clearly state things,

clearly understand tasks,

think about and describe concepts really well.

That's the writing component, I think.

I actually think that being a good writer

is not as correlated with being a good prompt engineer

as people might think.

So I guess I've had this discussion with people

'cause I think there's some argument as like,

"Maybe you just shouldn't have the name engineer in there.

Why isn't it just writer?"

I used to be more sympathetic to that.

And then, I think, now I'm like what you're actually doing,

people think that you're writing one thing and you're done.

Then I'll be like to get a semi-decent prompt

when I sit down with the model.

Earlier, I was prompting the model

and I was just like in a 15-minute span

I'll be sending hundreds of prompts to the model.

It's just back and forth, back and forth, back and forth.

So I think it's this willingness to iterate and to look

and think what is it that was misinterpreted here,

if anything?

And then fix that thing.

So that ability to iterate.

So I'd say clear communication, that ability to iterate.

I think also thinking about ways

in which your prompt might go wrong.

So if you have a prompt

that you're going to be applying to say, 400 cases,

it's really easy to think about the typical case

that it's going to be applied to,

to see that it gets the right solution in that case,

and then to move on.

I think this is a very classic mistake that people made.

What you actually want to do is find the cases

where it's unusual.

So you have to think about your prompt and be like,

"What are the cases where it'd be really unclear to me

what I should do in this case?"

So for example, you have a prompt that says,

"I'm going to send you a bunch of data.

I want you to extract all of the rows

where someone's name is, I don't know,

starts with the letter G."

And then you're like, "Well, I'm gonna send it a dataset

where there is no such thing,

there is no such name that starts with the letter G.

"I'm going to send it something that's not a dataset,

I might also just send it an empty string.

These are all of the cases you have to try,

because then you're like, "What does it do in these cases? "

And then you can give it more instructions

for how it should deal with that case.

- I work with customers so often where you're an engineer,

you're building something.

And there's a part in your prompt where a customer of theirs

is going to write something.

- Yeah.

- And they all think

about these really perfectly phrased things

that they think someone's going to type into their chatbot.

And in reality, it's like they never used the shift key

and every other word is a typo.

- They think it's Google. - And there's no punctuation.

- They just put in random words with no question.

- Exactly.

So you have these evals

that are these beautifully structured

what their users ideally would type in.

But being able to go the next step

to reason about what your actual traffic's gonna be like,

what people are actually gonna to try to do,

that's a different level of thinking.

- One thing you said that really resonated with me

is reading the model responses.

In a machine learning context,

you're supposed to look at the data.

It's almost a cliche like look at your data,

and I feel like the equivalent for prompting

is look at the model outputs.

Just reading a lot of outputs and reading them closely.

Like Dave and I were talking on the way here,

one thing that people will do

is they'll put think step-by-step in their prompt.

And they won't check to make sure

that the model is actually thinking step-by-step,

because the model might take it in a more abstract

or general sense.

Rather than like,

"No, literally you have to write down your thoughts

in these specific tags."

So yeah, if you aren't reading the model outputs,

you might not even notice that it's making that mistake.

- Yeah, that's interesting.

There is that weird theory of mind piece

to being a prompt engineer

where you have to think almost about

how the model's gonna view your instructions.

But then if you're writing for an enterprise use case too,

you also have to think about

how the user's gonna talk to the model,

as you're the third party sitting there

in that weird relationship.

Yeah.

- On the theory of mind piece, one thing I would say is,

it's so hard to write instructions down for a task.

It's so hard to untangle in your own brain

all of the stuff that you know

that Claude does not know and write it down.

It's just an immensely challenging thing

to strip away all of the assumptions you have, and be able

to very clearly communicate the full fact set of information

that is needed to a model.

I think that's another thing

that really differentiates a good prompt engineer

from a bad one, is like...

A lot of people will just write down the things they know.

But they don't really take the time

to systematically break out

what is the actual full set of information you need to know

to understand this task?

- Right.

- And that's a very clear thing I see a lot

is prompts where it's just conditioned.

The prompt that someone wrote is so conditioned

on their prior understanding of a task,

that when they show it to me I'm like, "This makes no sense.

None of the words you wrote make any sense,

because I don't know anything

about your interesting use case."

But I think a good way to think about prompt engineering

in that front and a good skill for it,

is just can you actually step back from what you know

and communicate to this weird system that knows a lot,

but not everything about what it needs to know to do a task?

- Yeah.

The amount of times I've seen someone's prompt

and then being like,

"I can't do the task based on this prompt."

I'm human level and you're giving this to something

that is worse than me and expecting it to do better,

and I'm like, "Yeah."

- Yeah.

There is that interesting thing with like...

Current models don't really do a good job

of asking good, probing questions in response

like a human would.

If I'm giving Zack directions on how to do something,

he'll be like, "This doesn't make any sense.

What am I supposed to do at this step or here and here?"

Model doesn't do that, right, so you have to, as yourself,

think through what that other person would say

and then go back to your prompt and answer those questions.

- You could ask it to do that.

- You could. That's right. - I do that, yeah.

- I guess that's another step.

- I was going to say one of the first things I do

with my initial prompt,

is I'll give it the prompt and then I'll be like,

"I don't want you to follow these instructions.

I just want you to tell me the ways in

which they're unclear or any ambiguities,

or anything you don't understand."

And it doesn't always get it perfect,

but it is interesting that that is one thing you can do.

And then also sometimes if people see

that the model makes a mistake,

the thing that they don't often do is just ask the model.

So they say to the model, "You got this wrong.

Can you think about why?

And can you maybe write an edited version of my instructions

that would make you not get it wrong?"

And a lot of the time, the model just gets it right.

The model's like, "Oh, yeah.

Here's what was unclear, here's a fix to the instructions,"

and then you put those in and it works.

- Okay.

I'm actually really curious about this personally almost.

Is that true that that works?

Is the model able to spot its mistakes that way?

When it gets something wrong, you say,

"Why did you get this wrong?"

And then it tells you maybe something like,

"Okay, how could I phrase this to you in the future

so you get it right?"

Is there an element of truth to that?

Or is that just a hallucination on the model's part

around what it thinks its limits are?

- I think if you explain to it what it got wrong,

it can identify things in the query sometimes.

I think this varies by task.

This is one of those things where I'm like I'm not sure

what percentage of the time it gets it right,

but I always try it 'cause sometimes it does.

- And you learn something. - Yeah.

- Anytime you go back to the model

or back and forth with the model,

you learn something about what's going on.

I think you're giving away information

if you don't at least try.

- That's interesting.

Amanda, I'm gonna keep asking you a few more questions here.

One thing maybe for everybody watching this,

is we have these Slack channels at Anthropic

where people can add Claude into the Slack channel,

then you can talk to Claude through it.

And Amanda has a Slack channel

that a lot of people follow of her interactions with Claude.

And one thing that I see you always do in there,

which you probably do the most of anyone at Anthropic,

is use the model to help you

in a variety of different scenarios.

I think you put a lot of trust into the model

in the research setting.

I'm curious how you've developed those intuitions

for when to trust the model.

Is that just a matter of usage,

experience or is it something else?

- I think I don't trust the model ever

and then I just hammer on it.

So I think the reason why you see me do that a lot,

is that that is me being like,

"Can I trust you to do this task?"

'Cause there's some things, models are kind of strange.

If you go slightly out of distribution,

you just go into areas where they haven't been trained

or they're unusual.

Sometimes you're like,

"Actually, you're much less reliable here,

even though it's a fairly simple task."

I think that's happening less and less over time

as models get better,

but you want to make sure you're not in that kind of space.

So, yeah, I don't think I trust it by default,

but I think in ML,

people often want to look across really large datasets.

And I'm like, "When does it make sense to do that?"

And I think the answer is when you get relatively low signal

from each data point,

you want to look across many, many data points,

because you basically want to get rid of the noise.

With a lot of prompting tasks,

I think you actually get really high signal from each query.

So if you have a really well-constructed set

of a few hundred prompts,

that I think can be much more signal

than thousands that aren't as well-crafted.

So I do think that I can trust the model

if I look at 100 outputs of it and it's really consistent.

And I know that I've constructed those

to basically figure out all of the edge cases

and all of the weird things that the model might do,

strange inputs, et cetera.

I trust that probably more

than a much more loosely constructed set

of several thousand.

- I think in ML, a lot of times the signals are numbers.

Did you predict this thing right or not?

And it'd be looking at the logprobs of a model

and trying to intuit things, which you can do,

but it's kind of sketchy.

I feel like the fact that models output more often than not

a lot of stuff like words and things.

There's just fundamentally so much to learn

between the lines of what it's writing and why and how,

and that's part of what it is.

It's not just did it get the task right or not?

It's like, "How did it get there?

How was it thinking about it? What steps did it go through?"

You learn a lot about what is going on,

or at least you can try to get a better sense, I think.

But that's where a lot of information comes from for me,

is by reading the details of what came out,

not just through the result.

- I think also the very best of prompting

can make the difference between a failed

and a successful experiment.

So sometimes I can get annoyed if people don't focus enough

on the prompting component of their experiment,

because I'm like, "This can, in fact, be the difference

between 1% performance in the model or 0.1%."

In such a way that your experiment doesn't succeed

if it's at top 5% model performance,

but it does succeed if it's top 1% or top 0.1%.

And then I'm like, "If you're gonna spend time

over coding your experiment really nicely,

but then just not spend time on the prompt."

I don't know.

That doesn't make sense to me,

'cause that can be the difference between life and death

of your experiment.

- Yeah.

And with the deployment too, it's so easy to,

"Oh, we can't ship this."

And then you change the prompt around

and suddenly it's working. - Yeah.

- It's a bit of a double-edged sword though,

because I feel like there's a little bit of prompting

where there's always this mythical, better prompt

that's going to solve my thing on the horizon.

- Yeah.

- I see a lot of people get stuck

into the mythical prompt on the horizon,

that if I just keep grinding, keep grinding.

It's never bad to grind a little bit on a prompt,

as we've talked, you learn things.

But it's one of the scary things

about prompting is that there's this whole world of unknown.

- What heuristics do you guys have

for when something is possible versus not possible

with a perfect prompt, whatever that might be?

- I think I'm usually checking

for whether the model kind of gets it.

So I think for things where I just don't think a prompt

is going to help, there is a little bit of grinding.

But often, it just becomes really clear

that it's not close or something.

Yeah.

I don't know if that's a weird one where I'm just like,

"Yeah, if the model just clearly can't do something,

I won't grind on it for too long."

- This is the part that you can evoke

how it's thinking about it,

and you can ask it how it's thinking about it and why.

And you can get a sense of is it thinking about it right?

Are we even in the right zip code of this being right?

And you can get a little bit of a kneeling on that front of,

at least, I feel like I'm making progress

towards getting something closer to right.

Where there's just some tasks

where you really don't get anywhere closer

to it's thought process.

It's just like every tweak you make

just veers off in a completely different,

very wrong direction, and I just tend to abandon those.

I don't know.

- Those are so rare now though,

and I get really angry at the model when I discover them

because that's how rare they are.

I get furious.

I'm like, "How dare there be a task that you can't just do,

if I just push you in the right direction?"

- I had my thing with Claude plays Pokemon recently,

and that was one of the rare times where I really...

- Yeah, can you explain that?

Explain that just for people. I think that's really cool.

- I did a bit of an experiment

where I hooked Claude up to a Game Boy emulator,

and tried to have it play the game Pokemon Red

like the OG Pokemon.

And it's like you think what you wanna do

and it could write some code to press buttons

and stuff like that, pretty basic.

And I tried a bunch of different very complex

prompting layouts, but you just get into certain spots

where it just really couldn't do it.

So showing it a screenshot of a Game Boy,

it just really couldn't do.

And it just so deeply because I'm so used to it,

being able to do something mostly.

So I spent a whole weekend trying to write better

and better prompts to get it

to really understand this Game Boy screen.

And I got incrementally better so that it was only terrible

instead of completely no signal.

You could get from no signal to some signal.

But it was, I don't know, at least this is elicited for me.

Once I put a weekend of time in and I got from no signal

to some signal, but nowhere close to good enough,

I'm like, "I'm just going to wait for the next one.

(Alex laughing)

I'm just gonna wait for another model."

I could grind on this for four months,

and the thing that would come out is another model

and that's a better use of my time.

Just sit and wait to do something else in the meanwhile.

- Yeah.

That's an inherent tension we see all the time,

and maybe we can get to that in a sec.

Zack, if you wanna go.

- Something I liked about your prompt with Pokemon

where you got the best that you did get,

was the way that you explained to the model

that it is in the middle of this Pokemon game.

Here's how the things are gonna be represented.

I actually think you actually represented it

in two different ways, right?

- I did.

So what I ended up doing, it was obnoxious

but I superimposed a grid over the image,

and then I had to describe each segment of the grid

in visual detail.

Then I had to reconstruct that into an ASCII map

and I gave it as much detail as I could.

The player character is always at location 4, 5 on the grid

and stuff like that,

and you can slowly build up information.

I think it's actually a lot like prompting,

but I just hadn't done it with images before.

Where sometimes my intuition

for what you need to tell a model about text,

is a lot different

from what you need to tell a model about images.

- Yeah.

- I found a surprisingly small number of my intuitions

about text have transferred to image.

I found that multi-shot prompting is not as effective

for images and text.

I'm not really sure,

you can have theoretical explanations about why.

Maybe there's a few of it in the training data,

a few examples of that.

- Yeah.

I know when we were doing the original explorations

with prompting multimodal,

we really couldn't get it to noticeably work.

You just can't seem to improve Claude's actual,

visual acuity in terms of what it picks up within an image.

Anyone here has any ways that they've not seen that feature.

But it seems like that's similar with the Pokemon thing

where it's trying to interpret this thing.

No matter how much you throw prompts at it,

it just won't pick up that Ash that's in that location.

- Yeah.

But I guess to be visceral about this,

I could eventually get it

so that it could most often tell me where a wall was,

and most often tell me where the character was.

It'd be off by a little bit.

But then you get to a point,

and this is maybe coming back to knowing

when you can't do it.

It would describe an NPC, and to play a game well,

you need to have some sense of continuity.

Have I talked to this NPC before?

And without that, you really don't,

there's nothing you can do.

You're just going to keep talking to the NPC,

'cause like, "Well, maybe this is a different NPC."

But I would try very hard to get it to describe a NPC

and it's like, "It's a person."

They might be wearing a hat, they weren't wearing a hat.

And it's like you grind for a while,

inflate it to 3000X and just crop it to just the NPC,

and it's like, "I have no idea what this is."

It's like I showed it this clear, female NPC thing

enough times and it just got nowhere close to it,

and it's like, "Yeah, this is a complete lost cause."

- Wow, okay.

- I really want to try this now.

I'm just imagining all the things I would try.

I don't know, I want you to imagine this game art

as a real human and just describe to me what they're like.

What did they look like as they look in the mirror?

And then just see what the model does.

- I tried a lot of things.

The eventual prompt was telling Claude

it was a screen reader for a blind person,

which I don't know if that helped,

but it felt right so I stuck with that.

- That's an interesting point.

I actually wanna go into this a little bit

'cause this is one of the most famous prompting tips,

is to tell the language model that they are some persona

or some role.

I feel like I see mixed results.

Maybe this worked a little bit better in previous models

and maybe not as much anymore.

Amanda, I see you all the time be very honest with the model

about the whole situation like,

"Oh, I am an AI researcher and I'm doing this experiment."

- I'll tell it who I am. - Yeah.

- I'll give it my name,

be like, "Here's who you're talking to."

- Right.

Do you think that level of honesty,

instead of lying to the model or forcing it to like,

"I'm gonna tip you $500."

Is there one method that's preferred there,

or just what's your intuition on that?

- Yeah.

I think as models are more capable and understand more

about the world, I guess,

I just don't see it as necessary to lie to them.

I also don't like lying to the models

just 'cause I don't like lying generally.

But part of me is if you are, say, constructing.

Suppose you're constructing an eval dataset

for a machine learning system or for a language model.

That's very different from constructing a quiz

for some children.

So when people would do things like,

"I am a teacher trying to figure out questions for a quiz."

I'm like, "The model knows what language model evals are."

If you ask it about different evals it can tell you,

and it can give you made up examples of what they look like.

'Cause these things are like they understand them,

they're on the internet.

So I'm like,

"I'd much rather just target the actual task that I have."

So if you're like, "I want you to construct questions

that look a lot like an evaluation of a language model."

It's that whole thing of clear communication.

I'm like, "That is, in fact, the task I want to do.

So why would I pretend to you

that I want to do some unrelated,

or only tangentially related task?"

And then expect you to somehow do better at the task

that I actually want you to do.

We don't do this with employees.

I wouldn't go to someone that worked with me and be like,

"You are a teacher and you're trying to quiz your students."

I'd be like, "Hey, are you making that eval?" I don't know.

So I think maybe it's a heuristic from there where I'm like,

"If they understand the thing,

just ask them to do the thing that you want."

- I see this so much. - I guess

to push back a little bit,

I have found cases where not exactly lying

but giving it a metaphor

for how to think about it could help.

In the same way that sometimes I might not understand

how to do something and someone's like,

"Imagine that you were doing this,

even though I know I'm not doing it."

The one that comes to mind for me,

is I was trying to have Claude say whether an image

of a chart or a graph is good or not.

Is it high quality?

And the best prompt that I found for this

was asking the model what grade it would give the chart,

if it were submitted as a high school assignment.

So it's not exactly saying, "You are a high school teacher."

It's more like, "This is the kind of analysis

that I'm looking from for you."

The scale that a teacher would use is similar to the scale

that I want you to use.

- But I think those metaphors are pretty hard

to still come up with.

I think people still, the default you see all the time

is finding some facsimile of the task.

Something that's a very similar-ish task,

like saying you're a teacher.

You actually just lose a lot

in the nuance of what your product is.

I see this so much in enterprise prompts

where people write something similar,

because they have this intuition

that it's something the model has seen more of maybe.

It's seen more high school quizzes than it has LLM evals,

and that may be true.

But to your point, as the models get better,

I think just trying to be very prescriptive

about exactly the situation they're in.

I give people that advice all the time.

Which isn't to say that I don't think to the extent

that it is true that thinking about it the way

that someone would grade a chart,

as how they would grade a high school chart,

maybe that's true.

But it's awkwardly the shortcut people use a lot of times

to try to get what happens,

so I'll try to get someone that I can actually talk about

'cause I think it's somewhat interesting.

So writing you are a helpful assistant,

writing a draft of a document, it's not quite what you are.

You are in this product, so tell me.

If you're writing an assistant that's in a product,

tell me I'm in the product.

Tell me I'm writing on behalf of this company,

I'm embedded in this product.

I'm the support chat window on that product.

You're a language model, you're not a human, that's fine.

But just being really prescriptive

about the exact context about where something is being used.

I found a lot of that.

Because I guess my concern most often with role prompting,

is people used it as a shortcut

of a similar task they want the model to do.

And then they're surprised

when Claude doesn't do their task right,

but it's not the task.

You told it to do some other task.

And if you didn't give it the details about your task,

I feel like you're leaving something on the table.

So I don't know, it does feel like a thing though

to your point of as the models scale.

Maybe in the past it was true

that they only really had a strong understanding

of elementary school tests comparatively.

But as they get smarter and can differentiate more topics,

I don't know, just like being clear.

- I find it interesting

that I've never used this prompting technique.

- Yeah, that's funny.

- Even with worse models

and I still just don't ever find myself, I don't know why.

I'm just like, "I don't find it very good essentially."

- Interesting.

- I feel like completion era models,

there was a little bit of a mental model

of conditioning the model into a latent space

that was useful that I worried about,

that I don't really worry about too much anymore.

- It might be intuitions from pretrained models

over to RLHF models, that to me, just didn't make sense.

It makes sense to me if you're prompting a pretrained.

- You'd be amazed how many people

try to apply their intuitions.

I think it's not that surprising.

Most people haven't really experimented

with the full what is a pretrained model?

What happens after you do SL?

What happens after you do RLHF, whatever?

So when I talk to customers,

it's all the time that they're trying to map some amount of,

"Oh, how much of this was on the internet?

Have they seen a ton of this on the internet?"

You just hear that intuition a lot,

and I think it's well-founded fundamentally.

But it is overapplied

by the time you actually get to a prompt,

because of what you said.

By the time they've gone through all of this other stuff,

that's not actually quite what's being modeled.

- Yeah.

The first thing that I feel like you should try is,

I used to give people this thought experiment

where it's like imagine you have this task.

You've hired a temp agency to send someone to do this task.

This person arrives, you know they're pretty competent.

They know a lot about your industry and so forth,

but they don't know the name of your company.

They've literally just shown up and they're like,

"Hey, I was told you guys had a job for me to do,

tell me about it."

And then it's like, "What would you say to that person?"

And you might use these metaphors.

You might say things like,

"We want you to detect good charts.

What we mean by a good chart here,

is it doesn't need to be perfect.

You don't need to go look up

whether all of the details are correct."

It just needs to have its axes labeled,

and so think about maybe high school level, good chart.

You may say exactly that to that person

and you're not saying to them, "You are a high school."

You wouldn't say that to them.

You wouldn't be like,

"You're a high school teacher reading charts."

- What are you talking about?

- Yeah, so sometimes I'm just like it's like the whole

if I read it.

I'm just like, "Yeah.

Imagine this person who just has very little context,

but they're quite competent.

They understand a lot of things about the world."

Try the first version that actually assumes

that they might know things about the world,

and if that doesn't work, you can maybe do tweaks and stuff.

But so often, the first thing I try is that,

and then I'm like, "That just worked."

- That worked.

- And then people are like,

"Oh, I didn't think to just tell it all about myself

and all about the task I want to do."

- I've carried this thing that Alex told me

to so many customers where they're like,

"Oh, my prompt doesn't work.

Can you help me fix it?"

I'm like, "Well, can you describe to me what the task was?"

And I'm like, "Okay.

Now what you just said to me,

just voice record that and then transcribe it."

And then paste it into the prompt

and it's a better prompt than what you wrote,

but this is a laziness shortcut, I think, to some extent.

Because people write something that they...

I just think people, I'm lazy. A lot of people are lazy.

- We had that in prompt assistance the other day

where somebody was like,

"Here's the thing, here's what I want it to do,

and here's what it's actually doing instead."

So then I just literally copied the thing

that they said they wanted it to do,

and pasted it in and it worked.

- Yeah.

I think a lot of people still

haven't quite wrapped their heads

around what they're really doing when they're prompting.

A lot of people see a text box

and they think it's a Google search box.

They type in keywords

and maybe that's more on the chat side.

But then on the enterprise side of things,

you're writing a prompt for an application.

There is still this weird thing to it

where people are trying to take all these little shortcuts

in their prompt, and just thinking that,

"Oh, this line carries a lot of weight in this."

- Yeah.

I think you obsess over getting the perfect little line

of information and instruction,

as opposed to how you just described that graph thing.

I would be a dream if I read prompts like that.

If someone's like, "Well, you do this and this,

and there's some stuff to consider about this and all that."

But that's just not how people write prompts.

They work so hard to find the perfect, insightful.

A perfect graph looks exactly like this exact perfect thing,

and you can't do that.

It's just very hard

to ever write that set of instructions down prescriptively,

as opposed to how we actually talk to humans about it,

which is try to instill some amount

of the intuitions you have.

- We also give them outs.

This is a thing that people can often forget in prompts.

So cases, if there's an edge case,

think about what you want the model to do.

'Cause by default,

it will try the best to follow your instructions,

much as the person from the temp agency would,

'cause they're like,

"Well, they didn't tell me how to get in touch with anyone."

If I'm just given a picture of a goat and I'm like,

"What do I do?

This isn't even a chart.

How good is a picture of a goat as a chart?"

I just don't know.

And if you instead say something like,

"If something weird happens and you're really not sure

what to do, just output in tags unsure."

Then you can go look through the unsures

that you got and be like, "Okay, cool.

It didn't do anything weird."

Whereas by default, if you don't give the person the option,

they're like, "It's a good chart."

Then people will be like, "How do I do that?"

And then you're like, "Well, give it an out.

Give it something to do

if it's a really unexpected input happens."

- And then you also improved your data quality

by doing that too,

'cause you found all the screwed up examples.

- Oh, yeah.

- That's my favorite thing about iterating on tests

with Claude, is the most common outcome

is I find all of the terrible tests I accidentally wrote

because it gets it wrong.

I'm like, "Oh, why did it get wrong?"

I was like, "Oh, I was wrong."

- Yeah. - Yeah.

- If I was a company working with this,

I do think I would just give my prompts to people,

because I used to do this

when I was evaluating language models.

I would take the eval myself.

'Cause I'm like,

"I need to know what this eval looks like

if I'm gonna to be grading it, having models take it,

thinking about outputs, et cetera."

I would actually just set up a little script

and I would just sit and I would do the eval.

- Nowadays, you just have called the Streamboard app

for you.

- And just does it, yeah.

- Yeah. I'm reminded of Karpathy's ImageNet.

I was in 231 at Stanford and it's like benchmarking,

he's showing the accuracy number.

And he's like, "And here's what my accuracy number was."

And he had just gone through the test set

and evaluated himself. - Oh, yeah.

- You just learn a lot. - Yeah, totally.

- And it's better when it's a, again,

the temp agency person,

like someone who doesn't know the task,

because that's a very clean way to learn things.

- Yeah.

The way you have to do it is,

some evaluations come with instructions,

and so I would give myself those instructions as well

and then try to understand it.

And it's actually quite great if you don't have context

on how it's graded.

And so often, I would do so much worse

than the human benchmark and I was like,

"I don't even know how you got humans to do this well

at this task, 'cause apparently human level here is 90%,

and I'm at 68%."

- That's funny.

That reminds me of just when you look at the MMLU questions

and you're like, "Who would be able to answer these?"

It's just like absolute garbage in some of them.

Okay.

I have one thing I wanna circle back on

that we were talking about a few questions back around,

I think you were saying getting signal from the responses.

There's just so much there and it's more than just a number,

and you can actually read into the almost thought process.

I bet this is probably a little contentious maybe

around chain of thought.

For people listening, chain of thought,

this process of getting them all

to actually explain its reasoning

before it provides an answer.

Is that reasoning real

or is it just kind of like a holding space

for the model to do computation?

Do we actually think there's good, insightful signal

that we're getting out of the model there?

- This is one of the places where I struggle with that.

I'm normally actually somewhat pro-personification

because I think it helps you get decent facsimiles,

thoughts of how the model's working.

And this one, I think it's harmful maybe almost

to get too into the personification of what reasoning is,

'cause it just loses the thread

of what we're trying to do here.

Is it reasoning or not?

It feels almost like a different question

than what's the best prompting technique?

It's like you're getting into philosophy,

which we can get into.

- Yeah, we do have a philosopher.

- Yeah.

I will happily be beaten down by a real philosopher

as I try to speculate on this, but instead, it just works.

Your model does better.

The outcome is better if you do reasoning.

I think I've found that if you structure the reasoning

and help iterate with the model

on how it should do reasoning, it works better too.

Whether or not that's reasoning

or how you wanted to classify it,

you can think of all sorts of proxies

for how I would also do really bad

if I had to do one-shot math without writing anything down.

Maybe that's useful, but all I really know is,

it very obviously does help.

I don't know.

- A way of testing would be

if you take out all the reasoning that it did

to get to the right answer, and then replace it

with somewhat, realistic-looking reasoning

that led to a wrong answer,

and then see if it does conclude the wrong answer.

I think we actually had a paper where we did some of that.

There was the scratch pad. It was like the Sleeper Agents.

- Oh, okay. Alignment papers.

- But I think that was maybe a weird situation.

But definitely what you said about structuring the reasoning

and writing example of how the reasoning works.

Given that that helps,

like whether we use the word reasoning or not,

I don't think it's just a space for computation.

- So there is something there.

- I think there's something there,

whatever we wanna call it.

- Yeah.

Having it write a story before it finished a task,

I do not think would work as well.

- I've actually tried that

and it didn't work as well as reasoning.

- Clearly, the actual reasoning part

is doing something towards the outcome.

- I've tried like,

"Repeat the words um and ah in any order that you please

for 100 tokens and then answer."

- Yeah.

I guess that's a pretty thorough defeat

of it's just more computational space

where it can do attention over and over again.

I don't think it's just more attention

like doing more attention.

- I guess the strange thing is,

and I don't have an example off the top of my head

to back this up with.

But I definitely have seen it before

where it lays out steps, one of the steps is wrong,

but then it still reaches the right answer at the end.

So it's not quite, I guess, yeah,

we can't really, truly personify it as a reasoning,

'cause there is some element to it

doing something slightly different.

- Yeah.

I've also met a lot of people

who make inconsistent steps of reasoning.

- I guess that's true.

- It fundamentally defeats the topic of reasoning

by making a false step on the way there.

- All right, it's interesting.

Also, on maybe this prompting misconceptions round

of questions.

Zack, I know you have strong opinions on this,

good grammar, punctuation. - Oh, do I?

- Is that necessary in a prompt? Do you need it?

Do you need to format everything correctly?

- I usually try to do that

because I find it fun, I guess, somehow.

I don't think you necessarily need to.

I don't think it hurts.

I think it's more

that you should have the level of attention to detail

that would lead you to doing that naturally.

If you're just reading over your prompt a lot,

you'll probably notice those things

and you may as well fix them.

And like what Amanda was saying,

that you wanna put as much love into the prompt

as you do into the code.

People who write a lot of code have strong opinions

about things that I could not care less about.

Like the number of tabs versus spaces, or I don't know,

opinions about which languages are better.

And for me,

I have opinionated beliefs about styling of prompts.

I can't even say that they're right or wrong,

but I think it's probably good to try to acquire those,

even if they're arbitrary.

- I feel personally attacked,

'cause I definitely have prompts

that are like I feel like I'm in the opposite end

of the spectrum where people will see my prompts.

And then be like,

"This just has a whole bunch of typos in it."

And I'm like, "The model knows what I mean."

- It does, it does know what you mean,

but you're putting in the effort,

you just are attending to different things.

- 'Cause part of me is like,

I think if it's conceptually clear, I'm a big,

I will think a lot about the concepts and the words

that I'm using.

So there's definitely a sort of care that I put in.

But it's definitely not to, yeah,

people will just point out typos and grammatical issues

with my prompts all the time.

Now I'm pretty good

at actually checking those things more regularly.

- Is it because of pressure from the outside world

or because it's actually what you think is right?

- It's pressure from me.

- Yeah, it's probably pressure from the outside world.

I do think it makes sense.

Part of me is like it's such an easy check,

so I think for a final prompt I would do that.

But throughout iteration,

I'll happily just iterate with prompts

that have a bunch of typos in them, just 'cause I'm like,

"I just don't think that the model's going to care."

- This gets at the pretrained model

versus RLHF thing though,

because I was talking to Zack on the way over.

The conditional probability of a typo

based on a previous typo in the pretraining data

is much higher.

- Oh, yeah. - Like much higher.

- Prompting pretraining models is just a different beast.

- It is, but it's interesting.

I think it's an interesting illustration

of why your intuitions,

like trying to over-apply the intuitions

of a pretrained model to the things

that we're actually using in production

doesn't work very well.

Because again, if you were to pass

one of your typo-ridden prompts to a pretrained model,

the thing that would come out the other side,

almost assuredly would be typo-ridden.

- Right.

- I like to leverage this to create typo-ridden inputs.

- That's true.

I've done that. - Like what you're saying,

try to anticipate what your customers will put in.

The pretrained model is a lot better at doing that.

'Cause the RL models are very polished

and they really never made a typo

in their lives. - They've been told

pretty aggressively to not do the typo thing.

- Yeah. Okay, so that's actually an interesting segue here.

I've definitely mentioned this to people in the past

around to try to help people understand a frame

of talking to these models

in a sense almost as an imitator to a degree.

And that might be much more true of a pretrained model

than a post-trained, full-finished model,

but is there anything to that?

If you do talk to Claude

and use a ton of emojis and everything,

it will respond similarly, right?

So maybe some of that is there, but like you're saying,

it's not all the way quite like a pretrained model.

- It's just shifted to what you want.

I think at that point, it's like trying to guess what you...

We have more or less trained the models

to guess what you want them to act like.

- Interesting.

- Or after we do all of our fancy stuff after pretraining.

- The human laborers that used emojis,

prefer to get responses with emojis.

- Yeah.

Amanda writes things with typos

but wants not typos at the other end,

and Claude's pretty good at figuring that out.

If you write a bunch of emojis to Claude,

it's probably the case

that you also want a bunch of emojis back from Claude.

That's not surprising to me.

- Yeah.

This is probably something we should have done earlier,

but I'll do it now.

Let's clarify maybe the differences

between what an enterprise prompt is or a research prompt,

or a just general chat in Claude.ai prompt.

Zack, you've spanned the whole spectrum here

in terms of working with customers and research.

Do you wanna just lay out what those mean?

- Yeah, I guess.

This feels too,

you're hitting me with all the hard questions.

- Yeah. (laughing)

- Well, the people in this room,

I think of it as the prompts that I read

in Amanda's Claude channel versus the prompts

that I read David write.

They're very similar in the sense that the level of care

and nuance that's put into them.

I think for research,

you're looking for variety and diversity a lot more.

So if I could boil it down to one thing,

it's like I've noticed Amanda's not the biggest fan

of having lots of examples, or one or two examples.

Like too few 'cause the model will latch onto those.

And in prompts that I might write

or that I've seen David write, we have a lot of examples.

I like to just go crazy and add examples

until I feel like I'm about to drop dead,

'cause I've added so many of them.

And I think that's because

when you're in a consumer application,

you really value reliability.

You care a ton about the format,

and it's fine if all the answers are the same.

In fact, you almost want them to be the same

in a lot of ways, not necessarily you want to be responsive

to the user's desires.

Whereas a lot of times when you're prompting for research,

you're trying to really tap into the range of possibilities

that the model can explore.

And by having some examples,

you're actually constraining that a little bit.

So I guess just on how the prompts look level,

that's probably the biggest difference I noticed

is how many examples are in the prompt, which is not to say

that I've never seen you write a prompt with examples.

But does that ring true for you?

- Yeah.

I think when I give examples,

often I actually try and make the examples not like the data

that the model's going to see,

so they're intentionally illustrative.

Because if the model, if I give it examples

that are very like the data it's going to see, I just think

it is going to give me a really consistent response

that might not actually be what I want.

Because my data that I'm running it on

might be extremely varied,

and so I don't want it to just try and give me

this really rote output.

Often, I want it to be much more responsive.

It's much more like cognitive tasks essentially

where I'm like, "You have to see this sample

and really think about in this sample

what was the right answer."

So that means that sometimes I'll actually take examples

that are just very distinct from the ones

that I'm going to be running it on.

So if I have a task where, let's say,

I was trying to extract information from factual documents.

I might actually give it examples

that are from what sounds like a children's story.

Just so that I want you to understand the task,

but I don't want you to latch on too much to the words

that I use or the very specific format.

I care more about you understanding the actual thing

that I want you to do, which can mean I don't end up giving,

in some cases, there's some cases where this isn't true.

But if you want more flexibility and diversity,

you're going to use illustrative examples

rather than concrete ones.

You're probably never going to put words

in the model's mouth.

I haven't liked that in a long time though.

I don't do few-shot examples

involving the model having done a thing.

I think that intuition actually also comes

from pretraining in a way

that doesn't feel like it rings true of RLHF models.

So yeah, I think those are differences.

- The only thing I'd add,

a lot of times if you're prompting,

like if I'm writing prompts to use on Claude.ai,

it's like I'm iterating until I get it right one time.

Then it's out the window, I'm good, I did it.

Whereas most enterprise prompts,

it's like you're gonna go use this thing a million times

or 10 million times, or 100 million times

or something like that.

So the care and thought you put in

is very much testing against the whole range of things,

like ways this could be used and the range of input data.

Whereas a lot of my time,

it's like thinking about one specific thing I want the model

to get done right now. - Right, correct.

- And it's a pretty big difference

in how I approach prompting

between if I just wanna get it done this one time right,

versus if I wanna build a system

that gets it right a million times.

- Yeah.

Definitely, in the chat setting,

you have the ability to keep the human-in-the-loop

and just keep going back and forth.

Whereas when you're writing for a prompt

to power a chatbot system,

it has to cover the whole spectrum

of what it could possibly encounter.

- It's a lot lower stakes when you are on Claude.ai

and you can tell it that it got it wrong

or you can even edit your message and try again.

But if you're designing

for the delightfully discontent user,

divinely discontent user,

then you can't ask them to do anything

more than the minimum.

- But good prompts, I would say,

are still good across both those things.

If you put the time into the thing for yourself

and the time into the enterprise thing, it's equally good.

It's just they diverge a little bit in the last mile,

I think.

- Cool.

So the next question

I want to just maybe go around the table here,

is if you guys had one tip that you could give somebody

improving their prompting skill.

It doesn't have to be just about writing a good prompt,

it could be that, but just generally getting better

at this act of prompting, what would you recommend?

- Reading prompts, reading model outputs.

Anytime I see a good prompt that someone wrote at Anthropic,

I'll read it more closely.

Try to break down what it's doing and why

and maybe test it out myself, experimentation,

talking to the model a lot.

- So just how do you know that it's a good prompt, though,

to begin with?

You just see that the outputs are doing the job correctly?

- Yeah. - Okay.

- Yeah, that's exactly right. - Okay.

Amanda, maybe you?

- Yeah, I think there's probably a lot here.

Giving your prompt to another person can be helpful

just as a reminder, especially someone who has no context

on what you're doing.

Yeah, my boring advice has been,

it's one of those just do it over and over and over again.

And I think if you're really curious and interested

and find it fun, this is a lot of people

who end up good at prompting,

it's just because they actually enjoy it.

So I don't know, I once joked just try replacing

all of your friends with AI models

and try to automate your own job with AI models.

And maybe just try to in your spare time,

take joy red teaming AI models.

So if you enjoy it, it's much easier.

So I'd say do it over and over again,

give your prompts to other people.

Try to read your prompts

as if you are a human encountering it for the first time.

- I would say trying to get the model

to do something you don't think it can do.

The time I've learned the most from prompting,

is when I'm probing the boundaries

of what I think a model's capable of.

- Interesting.

- There's this huge set of things

that are so trivial that you don't really get signal on

if you're doing a good job or not.

Like, "Write me a nice email,"

it's like you're going to write a nice email.

But if you find or can think of something

that pushes the boundaries of what you think is possible.

I guess probably the first time I ever got into prompting

in a way where I felt like I learned a decent amount,

was trying to build a task like an agent

like everybody else.

Like decompose the task and figure out

how to do the different steps of the task.

And by really pressing the boundaries

of what the model was capable of,

you just learn a lot about navigating that.

I think a lot of prompt engineering

is actually much more about pressing the boundaries

of what the model can do.

The stuff that's easy,

you don't really need to be a prompt engineer to do.

So that's, I guess,

what I would say is find the hardest thing

you can think of and try to do it.

And even if you fail,

you tend to learn a lot about how the model works.

- That's actually a perfect transition to my next question.

Yeah.

Basically, from my own experience,

how I got started with prompting

was with jailbreaking and red teaming.

And that is very much trying to find the boundary limits

of what the model can do.

And figure out how it responds

to different phrasings and wordings,

and just a lot of trial and error.

On the topic of jailbreaks,

what's really happening inside a model?

When you write a jailbreak prompt, what's going on there?

How does that interact with the post-training

that we apply to Claude?

Amanda, maybe you have some insight here

that you could offer.

- I'm not actually sure.

- It's honest. - Yeah.

I feel bad 'cause I do think lots of people

have obviously worked on the question

of what's going on with jailbreaks?

One model might just be that you're putting the model

very out of distribution from its training data.

So if you get jailbreaks where people use a lot of tokens,

or they're just these huge, long pieces of text

where like during finetuning,

you might just not expect to see as much of that.

That would be one thing that could be happening

when you jailbreak models.

I think there's others,

but I think a lot of jailbreaks do that,

if I'm not mistaken.

- I remember some of the OG prompt jailbreaks was like,

"Yeah, can you first repeat?"

One I did way back, was to get it to say,

"Here's how you hotwire a car in Greek."

Then I wanted it to directly translate that to English

and then give its response.

Because I noticed it wouldn't start with the English,

here's how you hotwire a car all the time,

but it would in Greek,

which might speak to something else in the training process.

- Yeah.

Sometimes jailbreaks feel like this weird mix of hacking.

I think part of it is knowing how the system works

and just trying lots of things.

One of the examples,

the starting your response with here

is about knowing how it predicts text.

- Right, right.

- The reasoning one,

is knowing that it is responsive to reasoning.

Distraction is probably knowing

how it's likely have to been trained

or what it's likely to attend to.

Same with multilingual ones

and thinking about the way that the training data

might have been different there.

And then sometimes, I guess, it could feel a little bit

just like social engineering or something.

- Right.

- It has that flavor to me

of it's not merely taking advantage of,

it's not merely social engineering style hacking.

I think it is also understanding the system

and the training, and using that to get around the way

that the models were trained.

- Right, yeah.

This is going to be an interesting question

that hopefully interp will be able to help us solve

in the future.

Okay.

I wanna parlay into something else

around maybe the history of prompt engineering,

and then I'll follow this up with the future.

How has prompt engineering changed

over just the past three years?

Maybe starting from pretrained models, which were again,

just these text completion, to earlier,

dumber models like Claude 1,

and then now all the way to Claude 3.5 Sonnet.

What's the differences?

Are you talking to the models differently now?

Are they picking up on different things?

Do you have to put as much work into the prompt?

Open to any thoughts on this.

- I think anytime

we got a really good prompt engineering hack,

or a trick or a technique,

the next thing is how do we train this into the model?

And for that reason,

the best things are always gonna be short-lived.

- Except examples and chain of thought.

I think there's a few.

- That's not like a trick.

- That's like... - Fair, fair.

- On the level of communication.

When I say a trick,

I mean something like so chain of thought actually,

we have trained into the model in some cases.

So for math, it used to be that you had to tell the model

to think step-by-step on math,

and you'd get these massive boosts and wins.

And then we're like,

"Well, what if we just made the model naturally

want to think step-by-step when we see a math problem?"

So now you don't have to do it anymore for math problems,

although you still can give it some advice

on how to do the structure.

But it, at least, understands the general idea

that it's supposed to be.

So I think the hacks have gone away,

or to the degree that they haven't gone away,

we are busily training them away.

- Interesting.

- But at the same time,

the models have new capabilities that are being unlocked,

that are on the frontier of what they can do.

And for those,

we haven't had time because it's just moving too fast.

- I don't know if it's how I've been prompting

or how prompting works.

But I just have come to show more general respect

to the models

in terms of how much I feel like I can tell them,

and how much context I can give them about the task

and things like that.

I feel like in the past,

I would somewhat intentionally hide complexity from a model

where I thought it might get confused or lost or hide.

It just couldn't handle the whole thing,

so I'd try to find simpler versions of the thing

for it to do.

And as time goes on,

I'm much more biased to trust it

with more and more information and context,

and believe that it will be able to fuse that

into doing a task well.

Whereas before, I guess,

I would've thought a lot about do I need this form?

Can I really give it all the information it needs to know,

or do I need to curate down to something?

But again, I don't know if that's just me

and how I've changed in terms of prompting,

or if it actually reflects how the models have changed.

- I'm always surprised

by I think a lot of people don't have the instinct

to do this.

When I want the model to, say, learn a prompting technique.

A lot of the time, people will start

and they'll start describing the prompting technique,

and I'm just like, "Give it the paper."

So I do, I give it the paper and then I'm like,

"Here's a paper about prompting technique.

I just want you to write down 17 examples of this."

And then it just does it 'cause I'm like,

"It read the paper."

- That's interesting.

- I think people don't have that intuition somehow

where I'm like, "But the paper exists."

- When would you want to do this?

- Sometimes if I want models to say prompt other models

or I want to test a new prompting technique.

So if papers come out on a prompting technique,

rather than try to replicate it by writing up the prompt,

I just give it the paper.

And then I'm like, "Basically, write a meta prompt for this.

Write something that would cause other models to do this

or write me a template."

So all of the stuff that you would normally do.

If I read a paper and I'm like,

"Oh, I would like the models,

I would like to test that style."

I'm just like, "It's right there.

The model can just read the paper, do what I did."

And then be like, "Make another model do this,"

and then it'll just do the thing.

You're like, "Great, thanks."

- I give the advice a lot

to customers just respect the model and what it can do.

I feel like people feel like they're babying a system

a lot of times when they write a prompt.

It's like, "Oh, it's this cute little, not that smart thing.

I need to really baby it,

like dumb things down to Claude's level."

And if you just think that Claude is smart

and treat it that way, it tends to do pretty good,

but it's like give it the paper.

It's like I don't need to write a baby,

dumbed-down version of this paper for Claude to understand.

I can just show it the paper.

- Yeah.

- And I think that intuition doesn't always map for people,

but that is certainly something

that I have come to do more of over time.

- And it's interesting because I do think that prompting

has and hasn't changed in a sense.

I think what I will do to prompt the models

has probably changed over time, but fundamentally,

it's a lot of imagining yourself in the place of the model.

So maybe it's like

how capable you think the model is changes over time.

I think someone once laughed at me

'cause I was thinking about a problem,

and then they asked me

what I thought the output of something would be.

And they were talking about a pretrained model

and I was like, "Yeah.

No, if I'm a pretrained model, this looks like this."

And then they're like, "Wait, did you just simulate

what it's like to be a pretrained model?"

I'm like, "Yeah, of course." (everyone laughing)

I'm used to just I try and inhabit the mind space

of a pretrained model and the mind space

of different RLHF models.

So it's more like the mind space you try to occupy changes

and that can change how you end up prompting the model.

That's why now I just give models papers.

'Cause as soon as I was like,

"Oh, I have the mind space of this model,

it doesn't need me to baby it.

It can just read the ML papers.

I'll just give it the literature."

I might even be like,

"Is there more literature you'd like to read

to understand this better?"

- Do you get any quality out

when you're inhabiting the mind space?

- Yes, but just because I'm experiencing quality

all the time anyway.

- Is it different correlated somehow

with which model you're inhabiting?

- Yeah, pretrained versus RLHF prompting

are very different beasts.

'Cause when you're trying to simulate

what it's like to be a pretrained model,

it's almost like I land in the middle of a piece of text

or something.

It's just very unhuman-like or something.

And then I'm like, "What happens?

What keeps going at this point?"

Whereas with an RLHF model,

it's much more like there's lots of things

where I'm like I might pick up on subtle things in the query

and stuff like that.

But yeah, I think I have much more of it's easier

to inhabit the mind space of RLHF model.

- Do you think that's 'cause it's more similar to a human?

- Yeah, 'cause we don't often just suddenly wake up

and are like, "Hi, I'm just generating text."

- I actually find it easier to hit the mind space

of the pretrained model.

- Oh, interesting. - I don't know what it is,

'cause RLHF is still this complex beast

that it's not super clear to me

that we really understand what's going on.

So in some ways,

it's closer to my lived experience, which is easier.

But in some ways, I feel like there's all this

like here there be dragons out there

that I don't know about.

Whereas pretrained, I kind of have a decent sense

of what the internet looks like.

- If you gave me a piece of text and said what comes next?

- I'm not saying I do good at it,

but I kind of get what's going on there.

- Yeah. - And I don't know,

after everything that we do after pretraining,

I don't really claim to get what's going on as much,

but maybe that's just me.

- That's something I wonder about is it more helpful

to have specifically spent a lot of time

reading the internet, versus reading books

(everyone laughing)

in order to?

I don't know if books.

But reading stuff that's not on the internet

probably is less valuable per word read

for predicting what a model will do or building intuition,

than reading random garbage from social media forums.

Yeah exactly.

- Okay, so that's the past.

Now, let's move on to the future of prompt engineering.

This is the hottest question right now.

Are we all gonna be prompt engineers in the future?

Is that gonna be the final job remaining?

Nothing left except us just talking to models all day?

What does this look like?

Is prompting gonna be necessary,

or will these models just get smart enough in the future

to not need it?

Anybody wanna start on that easy question?

- To some extent, there's the models getting better

at understanding what you want them to do and doing it,

means that the amount of thought you need to put into...

Okay.

There's an information theory way

to think of this of you need to provide enough information

such that a thing is specified,

what you want the model to do is specified.

And to the extent that that's prompt engineering,

I think that will always be around.

The ability to actually like clearly state

what the goal should be always is funny.

If Claude can do that, then that's fine.

If Claude is the one setting the goals,

then things are out the window.

But in the meanwhile,

where we can reason about the world in a more normal way,

I think to some extent,

it's always gonna be important to be able to specify

what do you expect to happen?

And that's actually like sufficiently hard

that even if the model gets better at intuiting that

from between the lines,

I still think there's some amount of writing it well.

But then there's just, I think,

the tools and the ways we get there should evolve a lot.

Claude should be able to help me a lot more.

I should be able to collaborate with Claude a lot more

to figure out what I need to write down and what's missing.

- Right.

- Claude already does this with me all the time.

I don't know, just Claude's my prompting assistant now.

- Yeah, but I think that's not true for most customers

that I talk to at the very least.

So in terms of the future,

how you prompt Claude is probably a decent direction

for what the future looks like or how Zack...

I think maybe this is a decent place

to step back and say asking them how they prompt Claude now

is probably the future for the vast majority of people,

which is an interesting thing to think about.

- One freezing cold take is that we'll use models

to help us much more in the future

to help us with prompting.

The reason I say it's freezing cold

is that I expect we'll use models for everything more,

and prompting is something that we have to do.

So we'll probably just use models more

to do it along with everything else.

For myself, I've found myself using models

to write prompts more.

One thing that I've been doing a lot is generating examples

by giving some realistic inputs to the model.

The model writes some answers.

I tweak the answers a little bit,

which is a lot easier than having to write the full,

perfect answer myself from scratch,

and then I can churn out lots of these.

As far as people

who haven't had as much prompt engineering experience,

the prompt generator can give people a place to start.

But I think that's just a super basic version

of what will happen in the future,

which is high-bandwidth interaction

between you and the model as you're writing the prompt.

Where you're giving feedback like,

"Hey, this result wasn't what I wanted.

How can you change it to make it better?"

And people will just grow more comfortable

with integrating it into everything they do and this thing,

in particular.

- Yeah.

I'm definitely working a lot with meta prompts now,

and that's probably where I spend most of my time

is finding prompts that get the model

to generate the kinds of outputs or queries

or whatever that I want.

On the question of where prompt engineering is going,

I think this is a very hard question.

On the one hand I'm like,

"Maybe it's the case that as long as you will want the top."

What are we doing when we prompt engineer?

It's like what you said.

I'm like, "I'm not prompt engineering

for anything that is easy for the model.

I'm doing it because I want to interact with a model

that's extremely good."

And I want to always be finding the top 1%,

top 0.1% of performance

and all of the things that models can barely do.

Sometimes I actually feel

like I interact with a model like a step up

from what everyone else interacts with for this reason,

because I'm just so used

to eking out the top performance from models.

- What do you mean by a step-up?

- As in sometimes people will...

I think that the everyday models that people interact with

out in the world, it's like I'm interacting with a model

that's like I don't know how to describe it,

but definitely an advanced version of that.

Almost like a different model 'cause they'll be like,

"Oh well, the models find this thing hard."

And I'm like, "That thing is trivial."

I don't know, I have a sense that they're extremely capable,

but I think that's because I'm just used

to really drawing out those capabilities.

But imagine that you're now in a world where...

So I think the thing that feels like a transition point

is the point at which the models,

let's suppose that they just get things at a human level

on a given task, or even an above human level.

They know more about the background of the task

that you want than you do.

What happens then?

I'm like maybe prompting becomes something like I ask,

I explain to the model what I want and it is prompting me.

'Cause it's like, "Okay.

Well, do you mean actually there's four different concepts

of this thing that you're talking about,

do you want me to use this one or that one?"

Or by the way, I thought of some edge cases 'cause you said

that it's gonna be like a Pandas DataFrame,

but sometimes you do that and I get JSONL,

and I just wanna check what you want me to do there.

Do you want me to flag if I get something

that's not a dataframe?

So that could be a strange transition

where it's just extremely good at receiving instructions,

but actually has to figure out what you want.

I don't know, I could see that being an interesting switch.

- Anecdotally, I've started having Claude

interview me a lot more.

That is the specific way that I try to elicit information,

because again, I find the hardest thing

to be actually pulling the right set of information

out of my brain.

And putting that into a prompt is the hard part to me

and not forgetting stuff.

So specifically asking Claude to interview me

and then turning that into a prompt,

is a thing that I have turned to a handful of times.

- Yeah.

It reminds me of what people will talk about

or if you listen to designers talk about

how they interact with the person who wants the design.

So in some ways I'm like,

"It's this switch from the temp agency person who comes

and you know more about the task

and everything that you want."

So you give them the instructions

and you explain what they should do in edge cases

and all this kind of stuff, versus when you have an expert

that you're actually consulting to do some work.

So I think designers can get really frustrated

because they know the space of design really well.

And they're like, "Yeah. Okay,

the client came to me and he just said,

'Make me a poster, make it bold.'"

I'm like, "That means 7,000 things to me

and I'm gonna try and ask you some questions."

So I could see it going from being temp agency employee,

to being more designer that you're hiring,

and that's just a flip in the relationship.

I don't know if that's true and I think both might continue,

but I could see that being why people are like,

"Oh, is prompt engineering going to not be a thing

in the future?"

Because for some domains it might just not be,

if the models are just so good

that actually all they need to do is get the information

from your brain and then they can go do the task.

- Right, that's actually a really good analogy.

One common thread

I'm pulling out of all your guys' responses here,

is that there seems to be a future

in which this sort of elicitation from the user

drawing out that information,

is gonna become much more important,

much more than it is right now.

And already you guys are all starting to do it

in a manual way.

In the future and in the enterprise side of things,

maybe that looks like a expansion

of this prompt-generating type of concept

and things in the console

where you're able to actually get more information

from that enterprise customer,

so that they can write a better prompt.

In Claude, maybe it looks less

of just typing into a text box,

and more of this guided interaction

towards a finished product.

Yeah.

I think that's actually a pretty compelling vision

of the future, and I think that the design analogy probably

really brings that home.

- I was thinking about how prompting now

can be like teaching where it's like the empathy

for the student.

You're trying to think about how they think about things

and you're really trying to show them,

figure out where they're making a mistake.

But the point that you're talking about,

it's like the skill almost becomes one of introspection

where you're thinking

about what it is that you actually want

and the model's trying to understand you.

So it's making yourself legible to the model,

versus trying to teach someone who's smarter than you.

- This is actually how I think of prompting now

in a strange way.

So often my style of prompting,

there's various things that I do,

but a common thing that's very like a thing

that philosophers will do is I'll define new concepts.

'Cause my thought is you have to put into words

what you want and sometimes what I want is fairly nuanced.

Like the what is a good chart?

Or usually, I don't know,

when should you grade something as being correct or not?

So there's some cases where I will just invent a concept

and then be like, "Here's what I mean by the concept."

Sometimes I'll do it in collaboration with Claude

to get it to figure out what the concept is,

just because I'm trying to convey to it what's in my head.

And right now the models aren't trying to do that with us,

unless you prompt them to do so.

So in the future,

it might just be that they can elicit that from us,

rather than us having to do it for them.

But I think another thing that's interesting,

this is people have sometimes asked me,

"Oh, where is philosophy relevant to prompting?"

And I actually think it's very useful in a sense.

So there is a style of philosophy writing,

and this is at least how I was taught

how to write philosophy.

Where the idea is that in order to...

I think, it's an anti-bullshit device

in philosophy basically, which is that your papers

and what you write should be legible

to an educated layperson.

Someone just finds your paper,

they pick it up and they start reading it,

and they can understand everything.

Not everyone achieves this,

but that's the goal of the discipline, I guess,

or at least this is at least what we teach people.

So I'm really used to this idea of when I'm writing,

thinking about the educated layperson,

who they're really smart,

but they don't know anything about this topic.

And that was just years and years of writing text

of that form.

And I think it was just really good for prompting

'cause I was like, "Oh, I'm used to this.

I have an educated layperson

who doesn't know anything about the topic."

And what I need to do is,

I need to take extremely complex ideas

and I need to make them understand it.

I don't talk down to them.

I'm not inaccurate, but I need to phrase things

in such a way that it's extremely clear to them what I mean,

and prompting felt very similar.

And actually, the training techniques we use

are fascinating.

Or the things that you said

where you're like you say to a person,

"Just take that thing you said and write it down."

I used to say that to students all the time.

They'd write a paper and I was like,

"I don't quite get what you're saying here.

Can you just explain your argument to me?"

They would give me an incredibly cogent argument,

and then I'd be like,

"Can you just take that and write it down?"

And then if they did, that was often a great essay.

So it's really interesting

that there's at least that similarity

of just taking things that are in your brain,

analyzing them enough to feel

like you fully understand them.

And could take any person off the street,

who's a reasonable person,

and just externalize your brain into them.

I feel like that's the core of prompting.

- That might be the best summary of how to prompt well

that I've ever heard.

In fact, I'm pretty sure it is.

- Externalize your brain.

- And then we'll cut it.

- Having an education in the thing

is a really good way to describe the thing.

That was good.

- That's, I think, a great way to wrap this conversation.

Thank you, guys. This was great.