Best of the Pod: Dwarkesh Patel’s Quest to Learn Everything

How is AI integrated into your work and

in your life right now?

So, a lot of topics I just find I have a

vague sense of what's happening and it's

super helpful to chat with Claude to

make sure I'm on the right track.

What is driving all of this? What do you

think that's about?

I really just want to know everything.

Wait, step back. Why is this necessary?

What's going on? How do I think about

the broader context of what's happening

here? Because I really can't ask good

questions unless I have a good mental

model of what they're talking about.

[Music]

This podcast is supported by Google. Hey

everyone, David here, one of the product

leads for Google Gemini. If you dream it

and describe it, V3 and Gemini can help

you bring it to life as a video now with

incredible sound effects, background

noise, and even dialogue. Try it with

the Google AI Pro plan or get the

highest access with the Ultra plan. Sign

up at gemini.google to get started and

show us what you create.

[Music]

Doresh, welcome to the show.

Thanks for having me, Dan.

I'm so excited to have you. For people

who don't know you, I assume everyone

knows you, but for people who don't, um,

you do the best most, like honestly the

smartest interviews in AI that I found.

Um, you have like really incredible

guests like Mark Zuckerberg, Dennis

Sabis, Patrick Collison. Um, you you

you've created like the go-to show for

smart people to to learn about AI, but

you also kind of branch out into lots of

other things like geopolitics and

history and stuff like that. It's it's

really great and you're like you're just

one of the one of the people that

inspire me to like make smart content.

So, I appreciate you coming on the show.

Yeah, that's very kind of you to say. I

mean, um, I've, uh, I've always been

sort of trying to make have the

conversations that I would like to have

if I was getting dinner with one of

these professors or CEOs. What would I

want to ask them? And, um, I'm I'm glad

other people enjoy them as well.

Yeah, it comes through. Um, and I I

think like it's it's really fun to get

to like turn turn the tables on you a

little bit because, uh, you've done some

interviews, but but mostly you're

interviewing other people. And I think

um like it's probably on people's minds

like how you use AI in your work in in

your life and so and so that's what

we're going to talk about today. So um

maybe just stop start by giving us a

little bit of an overview like um how is

AI integrated into your work and in your

life right now.

Yeah. So it's actually changed a lot. I

remember a year ago this was after I

think it was after GPT4. Um I somebody

asked me you know do you use AI to help

you with your research or prep? And I

was like not at all. It's completely

useless in mid. it gives you these banal

um you ask it like what should I ask so

and so professor and it'll give you

these banal where did you grow up what's

your book about whatever so initially it

was you know um terrible I think

recently the models have gotten to just

the point where with these like I don't

know 40 or especially with the cl new

cloud models they're intelligent and

interrogative and can consider the

context which you provide to them and so

they're still not that good at like what

should I ask this person because

obviously See, that's why I have a job,

right? So that I can come up with the

questions. But for the research itself

where you're, for me at least, I try to

like ingest everything they've ever

written. Um, all the rebuttals to their

ideas, all the other considerations, and

there's often a lot involved, especially

given there's like many different fields

I try to go deep into. Like the last

interview I just did was with Dylan

Patel, who writes semi analysis. It's a

um it's a publication about

semiconductors and AI hardware and so

on. So like there's a bunch you have to

learn and uh it it's I I mean I can go

through my workflow, but it's incredibly

useful to be able to like have this

thing where I'm like what's going on

here? Can you help me explain this? And

I guess one bigger thing I've been

thinking about is ever since I

interviewed Andy Matushak, if you've if

your audience is familiar, he's the guy

who did um who talks a lot about how

space repetition and other tools can

enhance our ability to learn and how the

normal mode of learning, you're actually

not picking up that much. If you pick up

a random book and start reading, you're

not getting that much out of it. And I

really have found that to very much to

be the case to the extent that if I'm

just like casually reading a book, I

think I'm basically wasting time or

entertaining myself. And I have come up

with a couple of um different workflows

and tools that help me really

interrogate and make sure I've

reinforced what I'm reading about or

learning. And the a tool like and

language model is very helpful cuz like

it gives it it gives you the contact um

the content in another context and you

can like it makes it can quiz you if you

want. So it's it's super helpful that

kind of stuff.

That's really cool. I think we should

start I want to start back to front like

with the the stuff you're using to read

because I think the reading all that

reading is sort of the input like one of

the inputs to the interviews and then

we'll get into the interviews. Um and

I'm really excited for both. So, so

let's start with with using AI to read

and to learn. the um you know so as I

was talking about like the one of the

main things I think is important is if

I'm studying a topic over the course of

a few weeks it's um especially if it's a

difficult topic it's like new to me it's

incredibly important that I've I'm not

just casually reading because if you're

just casually reading it's like every

day you're reread the same key terms the

same concepts and you're you start over

from scratch. Um, so one of the things I

like to do, for example, I was recently

interviewing Dylan, right? So if I go to

his publication, semi analysis, there's

just a ton of lingo and things you have

to understand. Um, so the new one was

pretty interesting. It's talking about

why nobody has built a huge training

cluster yet. And then first thing I do

is just like what are the key ideas and

concepts I really need to understand. So

I made myself um a hugging face. You

honestly don't need to do anything like

this. You can just it's pretty simple to

have claude build you hugging face space

or if you prefer it what it literally

does is like apply this prompt to

everything I paste in. So just you can

you just copy paste that prompt into

claude yourself. But basically it just

has I copy pasted some of the things in

Andy Matushek's post about how to write

good prompts and I just ask cla to make

those prompts for me space repetition

prompts. So um when I do this hopefully

in a few seconds we'll get something

back. um initially this will give just

give me some ideas of like what are the

key ideas here I need to understand so

um

super useful right I can even zoom in a

little bit so it's more helpful um so

for for the audience who's listening

like it's given me a bunch of question

answer pairs that consolidate the key

things I need to understand about this

post about um you know we can go through

the specifics here I'm sure that the

actual specifics of AI hardware will

bore But um a lot of the things where

it's like okay if you don't get this

you've like totally missed the boat

here. And so you can start with

something like this. I add it to my

space repetition app. Um or I can just

look through this and I'm getting a

sense of like oh okay here's what it

would take to train a GP4 level model on

a 100,000 H100 cluster. What are the

three main types of parallelism you need

to use to train on a big cluster or

whatever. And this is on for a technical

post. On other kinds of posts there

might be different kinds of cars that

come up for history. It might be a

different kind of thing. For philosophy,

it might be a different kind of thing.

So, this gives me a lay of the land.

I love this. This is super interesting.

I like I feel like I can go in like a

bunch of different directions, but what

where I want to start is like um how are

you reading and when are you reading?

So, is this like are you using this

specifically for um uh for reading that

you're doing for the show or are you

just doing this for any reading that

you're doing that you feel like is

serious and you really want to learn?

Um both. So,

just this weekend, I was reading um I

forgot the author's name, but it's a

book called Medieval Technology and

Social Change, and it's about how

different things that were developed

through um the last 1,00 to 500 years,

technologies, like the steerups, how

they affected society, and it's like you

can it's entertaining. You can read it.

And then

one of the things is like, okay, did I

really understand what's going on here

um with the relationship he's trying to

elucidate? So afterwards, in fact, I

have some claw chats where I was just

going through um while I was reading it,

um let's see if this recollects

do it. I I want to know I'm I'm on the

edge of my seat cuz I have this book.

It's like sitting on the desk in front

of me. Um and so I want to know what

what you got out of it.

Okay. So um first I was just asked to

make space repetition prompts for me.

First of all, I was just like I read the

chapter. I'm not sure I got it. So just

explain to me the chapter about how he

says that stirrups created feudalism.

Like what exactly was the connection

here? So it it's a much more condensed

like here's what's going on here.

Basically if you understand this it's a

useful scaffold so that when you're

reading the rest of the chapter you

understand where the pieces fit

together.

Have you tried like one one of the

things that's that I've tried with this

is like cuz sometimes it doesn't know

especially for a book like that where

it's like not that popular. Have you

tried like one one of the things I do is

create a little claude project and then

upload the text if I can find it. Have

you tried that?

In fact, let me just cl.ai

uh projects.

So, if I go to

um I literally just like I'm I'm a host

of a podcast where I try to ask good

questions. My upcoming guest is a

geneticist and I just upload the I get

the EPUB of the file. I convert the EPUB

to a text using an online converter. I

upload it to Project Knowledge. Then I

I've only just started prepping for this

guest, but I'll just have a bunch of

chats where I'm like, um, you know, how

how does he explain what groups made up

modern Europeans? It has all the context

in there. That's that ends up being

incredibly useful, like you were saying.

Yeah, that's so cool. I I I love that. I

love that feature. Okay. Wait, let's go

back to let's go back to stirrups and uh

and this chat you're having with uh um

uh with this book or about the book.

Yeah. So, you know, it explains that the

reason stirrups create a feudalism is

because you needed um you needed a lot

of land basically to support the kinds

of people who become heavy cavalry, the

knights. The knights need a lot of land

in order to um have the income to have,

you know, like uh armor and uh lances

and other kinds of equipment and to

train themselves. Um but you a knight is

only possible if you have a stir up

against which you can brace yourself as

you're attacking with a sword because

otherwise you're just like a Mongol

who's shooting bows and arrows. So, um,

but then then there's a bunch of stuff

that's confusing here, like why is it

that expensive to have be a knight that

you need to like completely confiscate

church lands in order to subsidize this

night lifestyle? Um, and then on these

kinds of questions, the author is dead,

but it's just like I'm just like murky

about it. I don't know what's going on.

So, I can just um these kinds of things

I can the book didn't even talk about,

right? But I can always just continue

the conversation with Claude and have it

explain what's going on. Um, and so this

is just like a recreational reading that

Claude ends up being super helpful with.

I think that's really interesting. What

What do you think about um like books

like this, like in your, you know, as a

as a person who likes history a lot,

books that sort of single out like a

specific thing like the stirrup and then

are like, well, you can trace all this

stuff to that like one thing where it

like makes so much sense. But then there

are things like I don't know like Guns,

Germs, and Steel where like Jared

Diamond had that whole thesis about I

can't remember the the exact thing but

it's like people in warmer climates or I

can't remember the exact theis but it

turned out to be like totally wrong.

Yeah. How do you feel about things like

that?

Yeah. So my opinion on these kinds of

books

there's

there's one I mean the sort of concise

answer is like yeah there's ones that do

it poorly but just don't read the ones

that do it poorly or something. There is

a failure mode for public intellectuals

where they initially start off with a

discipline and they do some exemplary

work there and then they write an

initial broad book that's about how this

idea explains a lot of the world and it

does incredibly well and now they're in

like public intellectual mode and now

that the next book has to be like here's

my theory of everything and it just not

that satisfying. So, I do worry about

those kinds of things. But, um,

presumably the reason I don't know. I'm

not I'm not into like reading 500page

books about like uh ju just how this

tear up physically worked. Like, what's

the point of that, right? I do want to

understand the implications. And maybe

they're wrong, but um

I mean, what else are we trying to do

here, right? Do we just care about maybe

you just intrinsically care about how

the stereo physically works there? I I I

will point out a couple of examples. So

there there's a lot of interesting

topics where you really can't get at the

heart of the matter without just

considering the whole story. And in

fact, so a couple of biographies

especially stand out in this way where

if you look at Ko's biography of LBJ or

Codkin's biography of Stalin, it's

basically a history of this 20th century

or in the case of Codkin even before the

20th century. Um, I think this uh KO

books on LBJ start off with the Comanche

raids on uh frontier settlers in the mid

19th century or something and it goes

through rural life in Texas, why

electrification was such a big deal, a

whole bunch of other things right now.

Um, so it's basically a history of the

20th century, but it has a very specific

point of view or a specific locus of

character that's moving the story along.

And I find those to be incredibly

helpful in getting a full picture of

what's going on in an era. Um there's a

couple other books where they really

aren't trying to write a theory of

everything. Like I don't think Carol's

trying to write about like what is the

history of the 20th century, but they

just can't help themselves. They feel

like you really cannot understand the

very specific topic I care about unless

I tell you everything about everything.

And you know like um Codkin story uh

biography of Stalin starts with the like

Bismar's career as a military general

and how that changed the way that

different powers thought about um

colonialism and the need to modernize

and that's where it starts right and

it's a biography of Stalin. So uh yeah I

love those kinds of books. I think

there's like a very just deep point

about the universe being interconnected

there, but there's also like a really

interesting point for um people who want

to make stuff like make writing or make

podcasts or whatever cuz like there's

this deep fear that everyone has about

like being pigeonhold and it's like well

if I pick this like really specific

topic I won't be able to like bring all

of myself to it or I won't I won't be

able to be like multifaceted and it's

like no no no if you just pick one guy

Lyndon Johnson and really get deep into

him you have to explain everything else

about the world in order to explain him.

And I I love that. And like as a creator

myself, like that's the thing that I

think about when I'm like, "Oh, maybe

I'm getting too narrow here." It's like,

"No, no, the narrow is actually good.

You can find the entire universe in the

narrow."

Yes. Yeah. Yeah. Yeah. I I couldn't have

said it better.

Yeah. Um Okay. So, so basically what I'm

what I'm seeing right now is you're

using Claude when you're reading books

that you care about, you care about like

learning from and you're using it a

little bit to like prepare your prepare

the mind your mind for like what you're

about to read, which I think is a

particularly good for like difficult

books or for like thinking through a

particular argument before you like go

through it. You're asking questions.

You're asking questions. So, you're kind

of like it's a reading companion. You're

getting more out of the books you read

from that. But then you kind of take

take your what you've read and throw it

into this um this uh card generator.

Yeah. And so that that's um mostly it's

just chatting with Claude. And so let me

see if I can find a better example. Um

so I mean a lot of topics I just find um

I' I've have a vague sense of what's

happening but I don't really like get

it. And it's super helpful to chat with

Claude to make sure I'm on the right

track. I one was like um

you know Dylan has a couple of posts

about how why packing as a technology is

super uh necessary for these advanced

chips. I'm not trying to make this

podcast all about AI hardware. It just

happens to be the last podcast I did. So

that's what you're getting. But um then

I'm I was it's like a confusing it's

like a five series post about how

advanced packing works and how you know

what what the technical specifications

are and I'm like wait step back what why

why is this necessary? What's going on?

Um all kinds of other questions about

when there's questions about how I'm

worried about where I might get too deep

in the weeds when I'm just explaining.

Yeah, basically I'm just like how how do

I think about the broader context of

what's happening here? because I really

can't ask good questions unless I have a

good mental model of um what they're

talking about. Like I really get where

all this is fits together.

That makes sense. And so and and and so

cloud is kind of like the first thing

you flip to when when you when you want

to know that. Are you using it on mobile

or using it on desktop?

Desktop.

Okay. Interesting. So you're doing most

of your like reading and research stuff

on desktop.

Yeah, that's right. H. And what do you

think about like uh just Claude being

really great right now and like chat

like I assume your CHBT usage is lower

than it used to be.

Yeah, I think these things will keep

getting better over time and

I

um I I you know I I I yeah I think just

like getting in the practice of using

these tools. I'll talk a little bit

about how these tools relate to my

post-production process. Initially it

was kind of useless but I did spend a

few weekends trying to write a few

prompts and key create a workflow. At

the time it was basically useless. Now

it's actually ended up being useful and

I can use the same you know Jupyter

notebooks or whatever to get things

done. So um it is worth investing in

getting even if they don't work

perfectly now to get them part of your

workflow so that as they keep getting

better you're getting the returns from

that.

Yeah that makes sense. So, so I want to

just go back to the uh the flat the the

the Anki card generator, the the space

repetition card generator. So, as part

of this, once you once you've done all

the sort of like conceptual um uh you

know, clearing the ground conceptually

for yourself to like kind of understand

the the basics of of what a guest is

talking about or an idea that you're

interested in, then you're kind of um

you're adding to your your flash card so

that I I guess so that it you retain the

information past even when you talk to

that guest. Is that right?

Yes, that's right. Um, I mean I I I

think of the larger mission of the

podcast is to consult. Why does the

podcast get better over time? And it's

because basically I'm getting smarter or

learning more things. I'm reducing my

ignorance around a bunch of topics. And

so to if I don't do that, I mean, I

think about all the episodes I did

before I interviewed Andy and started

using space repetition and I just like

really regret it because I talked to all

these world experts in a ton of

different domains. And to be honest,

like in many cases, I didn't take that

much away. I vaguely remember some

things. And now that I use I can walk

you through the kinds of cards I make

and the space operation tools I use, but

it's like totally a gamecher in terms of

what I can retain. And in fact, I think

it's not even about making sure I

remember what I discussed in a previous

episode or what I learned previously.

It's more about future learning because

I'm sure you've heard the saying about

um the uh you know learning compounds

because you can use what you've learned

in the past to um learn future things

because they all interconnect. Well, you

can't do that if you basically forgotten

most things you've learned in the past.

So I my learning has for future other

things has become much faster because I

have cached all these different um

concepts and figures and facts. So but

future things are just like I I

understand how everything fits together

much more. It's not even about the past.

It's really about future learning.

Can we see your uh I don't know what

what you use for for space repetitions.

Can we see your your deck?

Yep. I I will point out by the way as a

side notes one one use case of claw that

ended up actually being pretty useful

sometimes you read obscure phil like I

was reading Nick Lan's selected writings

about AI and his accelerationism and I

was like what's going on like I

genuinely I'm like what is his argument

basically like why does he think that

the AI takeover and the whatever thing

it creates in the aftermath will be goo

because he's a smart guy I'm assume he

has an interesting argument so I upload

the PDF of his selected writings I just

asked Cloud like Okay, so why does he

think it's a good thing that AIS take

over humans? And he they it offers a

summary like initially this isn't

necessarily that helpful cuz I kind of

did read this in the essay. But what's

what's helpful is that when you go

through and I'm like I respond like I

don't get it. Like what does he think is

wrong with human society that you can't

uh you have to erase it and then he

gives explanation. I'm like I still

don't get it. Like what exactly are you

talking about here? And then like here's

what I do with the podcast, right? have

the guest on and I ask them, "What do

you mean here? Like, I I I disagree.

Here's a contradiction, whatever." And

going through their writings with Claude

and like, "Have I have I actually found

a sort of blind spot in their thinking

or is this just me being confused about

their ideas?" is super helpful.

That's that is really interesting. It's

like um you can you can get you can get

down to a deeper level before you talk

to them so that you can start there with

them as opposed to like starting at the

surface.

Exactly.

Which is really cool. I I use that too

for like um for for difficult books like

not necessarily for like interviewing

the author of those books but like for

example I interviewed Reed Hoffman like

I don't know a month or two ago and I

wanted to talk to him about the kind of

like intersection between philosophy and

AI and he was like um he he like uh

almost became a philosophy professor

like at Oxford wanted to like was really

deep into Vickingstein so like I read a

bunch of Vickenstein um which I hadn't

read in a while and I I just used Claude

for it and it was like so much better.

Um because I like I haven't taken a

Vickenstein class or maybe I took one in

college like a long time ago, but I've

read him a lot and it just there are

always those those points in those kinds

of books where you're like I think I I

know what they're saying, but like I'd

probably have to go to a a grad graduate

school and like get a masters in this to

like really know. And Claude is actually

like makes me be like oh I don't need

that anymore. Like I I any book I want

to read like this like I basically know.

And it just helped me so much in that in

that interview because I could just ask

read like really deep Vickenstein

related questions and he could answer

them.

Yep. Yep. I think that's totally

legitimate. I think some people would be

like, "Oh, you need to read it in the

original blah blah blah." I think if you

care about the ideas and you think the

ideas are timeless and not the ideas are

not about the specific kind of pros that

the original author used, but just

generally like what is the essence and

the gist of what's happening here. If

you care about the ideas, and I think

this is totally valid, right? I I don't

just agree with the people who are like,

"No, you need to read like the specific

uh syllables that Wickenstein used."

Yeah. I mean, I'm also just saying like

I'm I have the book open and then I just

take one of his like statements and just

throw it in there and then it's like

here's here's what what it means or

whatever, which which I think is really

great. Um, okay. So, you're going to

you're going to show us the uh the the

space repetition card. So, what what app

is this?

This is Mochi. Um, interesting. It's

like an I this is the one I use. Um why

uh so that actually I don't have any

cards today because I just went through

them this morning but let me give you a

sense of let me give you a sense of what

kinds of things I have right so

I have um if you go through history

recently I've been planning on

interviewing David Reich who is a

geneticist who explores human origins

and these are especially cases where

just like reading the book I'm like I

would have totally forgotten he names

all these different ancestral groups and

how they combined and in what eras was,

you know, when did the Yamina people

come through Europe? When did the

Anatolian hunter gatherers, you know,

wash over uh Eurasia, all these things

that was just like you read it in one

year, it goes out the other one, unless

you make cards for it. And so I made a

ton of cards about this kind of stuff.

Um,

and you know, so there's examples of

that here. It's especially useful for

hardware and um technical things. So

here I feel like if I don't make cards,

I'm just constantly relearning the same

things because I didn't learn the lingo

the in the right way first. It's not

just about learning the terminology.

It's about understanding the underlying

concepts. Let me give you a good example

of that. So um maybe I I'll step back

and I'll explain like I go through these

cards in the morning. If I maybe you can

see what it kind of looks like if I do

the craards thing.

um I can go through them and

right now I'm like I remember this right

I remember it's u this is the first one

that came up randomly but it's um multi-

query uh attention to not have to use KV

huge KV values and then sharing KV

values between layers and using local

attention and that's the answer now um

it seems sort of trivial right now

because it's just like three things but

like I would have totally forgotten

about this if I hadn't made a card for

this as soon as I read the blog post.

And then it just like I've wasted my

time in the future if I'm learning about

these technologies in a different

context. I just like don't have the

connection to what was happening here to

connect it to. Right? If I go to a

different um category, if I go cram

cards, um this is the the white thing. I

would have totally forgotten about it if

I hadn't made these cards. Um I yeah,

I'm just a big fan right now. I I sort

of I've become a space repetition fanboy

these days. H

how do you think about like the the

usefulness of space repetition in a

world where like any of these questions

is possibly is like pretty much

answerable like with claude with like a

you know one one search.

Yes. So I think it's about not

necessarily remembering this information

but when a future thing comes in you

understand like the conceptual in fact

let me give you a good example of this

right. I remember

sometimes I actually make cards about

facts that I don't even understand in

the moment but in the future the as I

learn more about the field as a as a

sort of territory becomes more clear the

things I said in the card make more

sense to me. So if I um I was reading

some of Colin Burns papers and so I made

this card about like why Colin Burns

thinks that uh alignment is a tractable

problem or understanding what model

thinks is a tractable problem. And um at

the time I wrote things down about like

uh you know features are in a linear

space. What the what does that mean? Um

or like we can sort of see features in

other sorts of categories. And at the

time I'm like I have no idea what this

means. I'm just going to write it down

cuz I read the blog post and there was

no point of reading the blog post if I'm

not going to make the card. Later on, as

I learned more about how the residual

stream model of how attention works, you

know, how what that is and so forth.

This card made much more sense to me in

the future. But I would have just like

totally memory hold or not even memory

hold. I would totally forgotten this

content which required future

understanding if I hadn't made a card of

it. And then when I see the card again

in the future, I'm like, "Oh, this is

what Colin Burns meant now that I

understand how attention works. This is

what it means, right?" Um,

yeah,

this is this is really interesting to

me. So, I want to get into some of the

like the ways that you use AI for for

interview prep. Um, because I think we

we've mostly covered the reading stuff,

but before we do that, I just want to

like understand like what is driving all

of this? like it feels like you are just

consuming massive amounts of information

and turning that into knowledge in your

head. Like you have this in sort of just

overdrive of curiosity which I actually

resonate with a lot. Like I'm I'm

surrounded right now by books. Um and

I'm just sort of like um curious for you

what what that what do you think that's

about? Um

I think I um I've uh I I I I really just

want to know everything, right? It's I I

don't know how to express it. There's a

beautiful passage in a Will Durant book

as he's turning 90 where he's writing a

memoir basically of his main ideas um

called Fallen Leaves. And there's a

passage on philosophy where he says, you

know, as as you get older, maybe with

all the philosophy and history I've

done, I can I've reached some plateau of

higher understanding and clearer insight

or at least I've understood that such a

thing is possible. And

something like that just resonates with

me. I don't know. I just like I find

that idea really appealing. I'm nowhere

close to it, but I just hopefully in the

years to come that'll just be a thing

that um I also really admire people I've

had on the podcast who do have these

self-consistent and really deeply

interrogated world models. You know,

I've interviewed these guests and some

of them um people a couple names come to

mind, people like Carl Schulman or Tyler

Cowan or Burn Hobart. they it feels like

they've really read everything and you

know everything you know is a subset of

what they know and I just um I I I find

them to be super compelling as thinkers.

Of course there's many things they can

still be wrong about. I don't I'm not

one of these people who buy like there's

like a thing where you just know

everything and you can never be wrong,

right? you always have blind spots, but

the abil their ability to which you can

see when you talk to them to connect

anything you ask them about and they're

like clawed six in the sense of like you

start talking about why has a fraction

of finance as a percent of GDP. I

remember asking Tyler this and he has a

right off the cuff just super

interesting answer that connects a bunch

of different disciplines. um you asked

Carl about like how fast um AI hardware

could grow and just like done the sort

of firmy estimates on how fast algae

bloom and how much solar power they

consume and um how many fabs is making

it just like I I I find the sort of

compression of the input they've

ingested over their lives and they can

not only do they know that stuff but

they can really connect it in a really

interesting and compelling novel way. I

I I I find it super compelling.

And and in terms of developing your own

worldview, like do you have that

anywhere where it's like you're creating

like some sort of living document or is

it just all in your head like all the

stuff that you're learning? Obviously,

you have the you have the cards, but

that feels like more like um dots in the

space rather than like uh the ways that

they all connect and and how you think

about everything all together as a

system. Hm. I think um I've been trying

to do more of this recently. Uh and now

that I've sort of built up an underlying

maybe vocabulary or understanding

because of the podcast, it makes sense

to do more of this. Something I've been

doing recently, if let me pull this up.

I've only just started. Hopefully,

there'll be more by the time people are

looking at this, but um I've started

writing, you know, riffs on different

books or things I read. And it if I go

to um

it's basically on my website and um so

just like I can read a book and I have

questions or I connected with other

things I've read. I I remember for

example in when I was in Steven Pinker's

language instinct he was writing the

book before the fox P2 gene that can

help explain human uh language was

found. And so he has all these

observations that are then later

explained by the Fox P2 gene. Um, and so

I can just sort of the sort of

connection that you're talking about I

can do by riffing on other people's

ideas. I actually am curious. Do you

have suggestions on what I should be

doing? Maybe I should be writing more

blog posts or what do you suggest I

should do?

Um, well that's that's a good question.

So uh well before we get there like one

of the things that this reminds me of I

think Claude is so good for reading old

science books because it can tell you

what's outdated and what's what's not.

Um, I do that all the time and I I love

that. I love that little thing. But

yeah, I mean I think basically like

developing a worldview is like you have

to just try,

you know. Um, and you try over and over

and over again. And I I do think like

blog posts are uh really good for that,

especially like, you know, for me like I

have to write every week. Um, and so I'm

like forced to take a view on something.

And generally, if you're like

intellectually honest, you like want one

post to like somewhat like agree with

the last post and your audience will

call you out if you're like just

disagreeing with yourself all the time.

Um, so you're kind of developing a

worldview that way. But for me, like

right now, I'm actually like my big

thing this um this quarter is like I I

just have these like ideas that are

simmering that are like sort of the

relationship between like um language

models and like some deep philosophical

questions that we've been like talking

about since like Plato, which is like

the appearance, reality distinction and

like how do we know what's true and

what's knowledge and all that kind of

stuff. Like I think there's a lot of

overlap there and it requires like it's

going to be like a 10,000word post or

something like that. Um, and so what I'm

doing is I just have like a Claude

project with like I have all these like

little notes and riffs and like and

stuff and I'm just like going into

Claude and being like, "Hey, like

what's what's the thread here? Like

what's going on? Can you help me like

figure out like there's something in me

that I have all these little ideas for,

but I can't quite like put it into an

argument that all makes sense?" And I

think just honestly like sitting with

that for like a couple months I will I

will know what's in there, but there's

something in there.

Um and yeah, it's cool.

Do do you make a clawed project to like

here's some the things I'm thinking, how

do they connect or like how do you keep

track of those things as over those

months?

Exactly. I I mean I'll just show it to

you. Um uh let's let me just pull it up.

So, okay. So, if I go into claude,

um, I have a couple different projects.

Um, one project is seeing like a

language model, which is the title of

this big post, whatever it is. Another

is Zen in the art motor maintenance. So,

this is like a book that I'm reading as

prep for this um this piece that I'm

writing. Um, I've read it a bunch of

times before, but like now I'm like

doing a little bit of a deeper read. And

so I have like the I have the uh the

whole book uploaded and then I can like

ask questions. Then I have another one

that I love called my psychology which

has a bunch of like journal entries uh

goals I've set for myself um over the

years. And then also like things I've

observed about my psychology or things

I'm working on like little aspects of

myself that I'd like to grow or change.

And so when I'm making decisions or

thinking something through, I just go in

there and it can reference all that

stuff so it knows who I am. Uh which is

really cool. Um,

so in seeing like a language model, let

me see if I could pull it up in the

projects uh directory.

Um, so I have like basically I have this

one note in Apple Notes, which is like

uh every time I have a little thing come

into my head, I'm like I just I just put

it in there. Like let me see if I can

find it for you. Um

um I just throw it in here. And this is

like huge and messy and it's like

different quotes from different books

and like just different ideas that like

come to me off the top of my head as I'm

like walking around. And I think that

there's like a thread here in all of

this stuff. Um they're all like I can

see how they're all related, but but

like I can't quite pull it out. And so

like what I've been doing is I just like

throw it all throw it all in in here. We

have uh we have this like all the all

the quotes and all the ideas and

fragments. I have a little bit of a

draft, like an intro, and then I have a

chapter of a book um uh by Richard Rory

that I think is is is really good um

called Pragmatism as anti-

athoritarianism that like kind of

sparked this whole thing. Like I read

that I read an art I read a chapter of

that book and then I was like suddenly

like rereading like a bunch of Plato and

like Aristotle and and like I was just

down this huge rabbit hole. And so um

like what I did for example is I put all

that stuff in here and I was like hey I

have a bunch of notes and some fragments

for of text for a long 10,000ish word

piece I want to write called seeing like

a language model but I need to

understand what I actually think and

make a bit of an outline before I get

started. In order to do that I need to

understand the patterns of what I've

been thinking and writing down and

reading about can you suggest some ways

that you can help me do this. I want to

get from where I am to an outline. you

have access to some fragments, notes,

and early unfinished intro. And it just

like has a bunch of ideas like thematic

analysis or argument mapping or

chronological development. And I'm just

like sort of going down the rabbit hole

with it where it's like um you know, I

asked it to do concept clustering. So

it's like, you know, one of the concepts

that I'm playing with is the

philosophical divide, Plato versus

Aristotle, which I think is like not

quite right. It's actually Plato versus

the Sophists, but like it's it's it's

close or the evolution of Western

thought. It's like how does Plato like

um uh uh how does Plato ladder up into

the rest of Western thought and and into

science and into just the way the

western mind works? Um and then how how

do language models sort of like differ

from that from that um paradigm. Um, so

that that's the that's the basic thing

that I'm trying for cuz I I I I do have

the reason I ask this question is cuz

I'm selfishly like I feel like a little

bit I haven't done the like big idea

thing as much as I really want to cuz

I'm I am writing every week. I am sort

of like reacting to stuff and so I want

to be a little bit more thoughtful. Um,

and this is this is my like attempt to

like put all of it together into

something that like makes sense.

Yeah. you know, as you're going through

this, this really actually makes me want

to write more because now that you're

talking about it, now that you asked the

question, I'm like, yeah, I I should be

sort of consolidating the things I'm

learning in a more comprehensive way.

And in a way, that's also more useful

and accessible to other people as well,

right? I, you know, I spend weeks like

learning about some random what, not

random, but like the things I care

about. I'm about to prepare for Daniel

Jorgen, the guy who wrote the prize.

this a history of oil or a geneticist um

you know AI research or whatever to the

extent that I'm getting something out of

these research processes I should

consolidated in a way that's not evident

in the podcast itself.

Yeah. I mean I I selfishly want you to

do that cuz I'm curious what you think.

Yeah. I appreciate that. You will send

this out through your newsletter, right?

Is is that the main

Yeah.

Yeah. Yeah. This this will go through

through newsletter. I might do something

special for it like um you know maybe

maybe I'll make a little mini site for

it when it when it launches but that's

sort of in the future. I have to

actually I have to actually write it

first. Um so okay so so so let's I want

to move on. I want to I want to talk

about the uh how how you use AI to to

sort of like do do the interview prep.

Um so so let's let's move into that and

then we can also um

uh we can also um maybe even like prep

for an interview together.

Yeah. Yep. Okay. Let's do it. Um I

honestly the interview prep is like it

requires a lot of work but fundamentally

what's happening is not that

complicated. Like I can just show you a

document I might have made in the past.

I'll share my screen. So honestly it

literally is just like I'm going through

I come up with a bunch of questions and

I sort of group them together in uh

relevant categories or if I go to um if

I was interviewing Dylan Patel sorry

this is not the right one. Um

just like a bunch of different yeah just

like a list of questions basically. It's

not that complicated, but the process of

coming up up with them is, you know,

very research intensive. So, we can go

through like if I'm I guess I'm like

only barely started preparing for, we

can go through the process preparing for

them.

Yeah. Can can we can we I just want to

stop at those questions like again

selfishly because I think it's really

interesting like you have these like

long lists of questions that are

organized by theme. Are you like going

down the list or are you sort of jumping

around to

bed? It's really interesting because um

I come up with these list of questions,

but it's like it really never ends up

being I ask question one and I ask

question two and I ask question three.

the it I you know I start off with an

interesting question and if you listen

to the interviews hopefully it comes off

more as a almost conversation because I

spend so much time preparing that I have

these questions basically memorized and

so the next one that is appropriate to

their response if they say something

about um you know me memorization in LMS

I'll have a question prepared about that

or related to that and I'll I'll just

ask it next because that's what fits in

together and so you know I'll have a

list and this is what I'll send them if

they ask for it. But like really just

sort of me off the cuff like here's

here's a question I remember that was

relevant to this in in the actual

interview.

That makes sense. So like the the point

of the doc is like it's almost like

writing the doc is is the prep itself

and it's you don't even necessarily need

it in the interview. Like maybe you have

it just in case. But yeah, that makes a

lot of sense. Yeah.

Um and then yeah, we we can even go

through let's see um I'm doing a couple

of interviews in the future. uh David

Reich and David Reich and Daniel Jurgen.

So the one the performer former is a

geneticist uh with about human origins.

The second wrote the prize which is the

you know the famous book about the

history of oil. Which one sounds more

interesting to you? We can do that

whichever one.

Uh I want to I want to do the famous

geneticists.

So let's go to Claude. In fact I I I do

have his book uploaded as a project. So

we can just use that.

That's great. And so basically what

we're going to do is like we're going to

we're going to watch you and I'll I'll

do it with you. We're going to prep for

an interview uh with this guy. What's

his name again?

Um David Reich.

David Reich. Okay, cool. Can we get like

a little bit of background on David

Reich? Like maybe we can even ask Claude

cuz like I'm you know obviously I'm a

I'm a newbie to David Reich's work. I

don't know. He is a geneticist at

Harvard and over the last decade or so

their research into how have human

populations across the world been formed

basically like how do the who are the

Europeans what groups make them up what

ancient migrations and genocides and

population replacements made them same

with the Indians or Native Americans or

Africans um it's completely changed uh I

mean they've basically sort of like made

many academic disciplines irrelevant

because they actually have empirical

data on like here's actually what

historically happened. You guys are

completely wrong about what you think

your theories of what happened. Um if

you're familiar with Nat Freeman's

challenge, you know, like you you have

these like burnt up squirrels but with

some advanced techniques you can get

some useful information out of them. I

feel it's in a similar vein. Obviously,

they're not the same kind of project,

but like it's a similar vein of like

once we develop the advanced mathematics

or genetics or whatever to understand

what's latent in the genome, um we've

like we've just uncovered a ton of

insight about what's been going on in

human history basically. And

sorry, I'm just going getting nerd

sniped and just going on riffs here.

Like well one of the interesting things

is uh you can see when one population

replaces another whether it was just

like oh we met and like we're like now

intermingling in trading and whatever or

is it like we're committing genocide

against you and you can tell that

because if in the case where it's

genocide or population replacement it

will be that the um the male line of the

population that is invading will um

overtake the uh male line of the

existing population but the female brine

will remain. So mitochondrial DNA is

only comes up in the female line and

you'll see like the female line because

they're getting you know like

the the new men who are coming in are

taking them as wives or something and

then the Anyway, so you can just like

learn a lot about like what kind of

invasion was it? Did they like conquer

or was they were they just like mingling

or something? Um one of the many things

you can see from the DNA.

That's really interesting.

Wait. And so so this is like basically

re-examining DNA evidence of like old

settlements and like basically and and

he's uncovering new ways of being able

to analyze the DNA like what's the

what's the what's the new methods that

they're using to like draw new

conclusions from existing evidence. One

of them is just that, right? Like seeing

how the Y chromosome and the

mitochondrial DNA cuz like you can just

learn a lot about a population based on

how the female versus male male line is

propagated about like what was the

social structure like and so forth.

Another is you can even tell the level

of inequality in a society because if

there's a lot so for example in India um

one of the things that was super

surprising is that the the amount of

indogamy which is to say that the um a

certain cast in a certain village would

just like not there wouldn't be any sort

of intermixing with another cast in a

neighboring village like it to the

extent that's true of nowhere else in

the world and they were able to find

this in India where the amount ount of

um social stratification. You can see

that in the genetic

catalog over the last thousands of years

where for thousands of years these two

neighboring casts haven't mixed um with

like 99% or something which is like even

from sort of infidelity or rape or

something you uh you would expect there

to be more than what actually ends up

being the case. So you can understand

modern uh culture in India based on what

has happened over the last few thousand

years.

That's really interesting. So, I want to

like I feel like you're doing like such

a good job of summarizing his main

ideas. Um, but I kind of wanted I kind

of want you to do the same thing with

with Claude so we can see how see how

you stack up versus Claude cuz obviously

you've you've you've you've input his

his book into into this project. So, it

has it has that as ref as reference

material. Can we ask it to just like

summarize like a few of his main ideas?

Yeah. Can you summarize

and maybe like the techniques he used to

come up with with

this

perfect new so what you're writing is

can you summarize the main ideas from

the book and the techniques you use to

come up with new insights

cool and one thing that's like really

cool about this is like you've been able

to do something like this with chbt for

a long time but ch's context window

isn't that long and So, it chops it up.

Um, and like it's not going to it's not

going to really be able to summarize the

entire thing because of that. You know,

it has to like find the right parts of

the parts of the book and the embedding

search in it is not very good and all

that kind of stuff. And Claude, you can

just like throw us throw a ton of stuff

in the context window and that just like

makes a big difference. Um, okay. So, it

looks like it looks like we've got some

some uh some answers.

So, it tells us that the ancient DNA is

revolutionized understanding of human

prehistory. Um and then we we've learned

that populations today are the result of

multiple waves of migration and mixture.

Uh um and then you know just like a

bunch of other genetic stuff. Um then it

talks about the key techniques about

whole genome sequencing and how they've

enabled these sorts of new new

discoveries they've been making. Um

yeah. Yeah. But but any anyways there's

a bunch of interesting things uh about

uh their research. Well, now I'm

interested in like, okay, so the key

techniques that it's using are whole

genome sequencing of ancient DNA

samples. So, is whole genome sequencing

like a a a new thing that you can do it

on ancient DNA samples? So, it's saying

like by improved extraction and

sequencing technologies is that that's

like

that is an interesting question. So, we

can even ask Cla because I'm not sure um

how uh how exactly do you

sequence an ancient

um or you know a prehistoric

uh genome like can you do like what what

is how does that work? Right.

Okay. So, they grind the bone and they

have then techniques to get the DNA out

of that. Now, another thing we can ask

is like um one thing I'm curious about.

Let's see. Uh I don't really remember

the chapter on um

Native Americans. I could ask about uh

what exactly happened with Native.

Here's one thing I'm curious about. like

um how would okay so I don't even know

if David Reich addresses this himself

but like how would David Reich's

theories

help explain why civilization

suddenly

um emerges

so rapidly

and that too

um concurrently

in the new and the old world

after 10,000 BC, aka the end of the last

ice age.

And then I maybe I'll just like ask

Claude why I think it's an interesting

question. So like this seems like a

really remarkable coincidence given how

long humans have been around, you know?

That is interesting. Well, coincidence

given that human correct my spelling

given that humans

um have been around for hundreds of

thousands of years.

I didn't realize it that we believe that

it emerged like at the same time in diff

in geographically disperate places.

That's totally new to me. I thought it I

thought it was like just in Mesopotamia.

No, it's actually there's a really good

book by um Peter Jackson called The

Great Divide and it's one of the most

interesting books I read. Just as a side

note, it's about Yeah, it's comparing

the emergence of civilization in the new

world versus your old world. So in the

old world um uh in the sorry in the new

world the um coral is like a

civilization in 3000 B.CE

and it's based on fishing and um not on

like conventional agriculture like memia

and he talks about how that changed the

evolution of the culture in the new

world versus the old world. But anyways

um

uh

that's really interesting.

Okay, so major population movements and

uh mixtures