The AI movement to end humanity | The Gray Area

Classical humanism has not actually done a good job of preparing us for the 21st century because we face this question that the successionists are right to

point out which is how do we know how we should and shouldn't use tech to evolve our species. This

our species. This is the gray area. I am Sean Illing. My

guest today is Sagal Samuel. She's a

writer here at Vox who covers AI ethics.

I invited Seagal on the show because she wrote an interesting and frankly kind of disturbing piece about AI successionism,

which is the idea that AI shouldn't just serve humanity, it should well succeed us. And this is not some fringe internet

us. And this is not some fringe internet fantasy. A lot of people in positions of

fantasy. A lot of people in positions of power and influence actually believe it.

So, G Samuel, welcome to the show.

Thanks. It's good to be with you.

Let's just jump right into it. Um, tell

me what AI successionism is and we will go from there.

Sure. Um,

the AI successionists are people who basically uh want AI to replace humanity. Like to

most people hearing that, that sounds insanely out there. Um, and so much of our like AI discourse is premised on no like protect humanity from the AI. The

successionists say like no, like we actually want AI to replace humanity because what if the AI is better? Um,

and they they have this image they use of the flame and the torch. They talk

about the flame of consciousness for the you know the potentially most valuable thing in the universe consciousness maybe it's very rare in the universe we don't know um and humanity they say is

just one torch carrying that flame and the torch is not really what matters right what matters is the flame and if AI is better at spreading the flame of consciousness across the universe and

can even experience like more blissful types of consciousness and higher forms of moral value than we ever could then let it rip, right? Just just let the AI

uh go ahead and be a worthy successor to humanity. That's how they phrase it.

humanity. That's how they phrase it.

I am practicing enormous restraint right now because there is so much about this that makes my head explode. But we will get there. I'm a professional. We will

get there. I'm a professional. We will

get there.

You're a professional, Sean. I have

faith in you.

Yeah. Thank you. So just to get this straight, these successionists, do they think AI

replacing humanity is inevitable or desirable or both?

Um, many of them think both. So it's

common to hear AI successionists say, look, AI is going to be our worthy successor. It is desirable for them to

successor. It is desirable for them to succeed us, and it's also inevitable. So

there's no point resisting it.

Resistance is futile. You might as well get on board and let it rip.

Roger that.

How you doing there, Sean? You you you having an aneurysm?

I'm I'm I'm being professional.

Uh-huh.

You went to an invite only symposium where these people who are real, who really believe this stuff were there.

What was that like?

It was really interesting because here's the thing.

It's very easy to be like really tongue and cheek about these people and write them off and be like gh this is just insane whatever you know but a we actually need to pay attention to them

because they are highly influential some of the people in the room at this symposium uh these are people at the top who work at the top AI labs people from deep mind

people from uh XAI you know people from all of these like top AI labs and people who work for think tanks that directly

influence the US government's policy on AI. Um, so these are not fringe

AI. Um, so these are not fringe characters. These are very important

characters. These are very important players in the story of how AI is going to roll out. But the most important or sort of the most interesting for me part

of the conference came after it ended, which like this is a good reporting trick is like the juiciest stuff always comes after the event, right? So you tag along for drinks at like the hotel bar

where people go after the conference and they're like a little more chill and then you hear what they really think and people just started to speak very freely and you know some would say yep let's

merge with AI let's let AI figure out how to do the merger and then take it off the leash and let it run the show.

Some people said forget merger I'm like a nothing little human. I don't even need need to be part of this like AI that goes forward. It's fine if humanity

is just completely wiped out. I take

comfort in the fact that these AIs were trained on human text. So, in some sense, the human spirit lives on through the AI, but wipe me out. I don't care.

Just let the AI take the reigns.

That is a pretty stunning conversation to hear over cocktails while you're like looking out over the Manhattan skyline.

um uh amidst this group of people who think they are the select individuals who might make it through that transformation or if they think AI will just wipe us all out are happy to make

that transformation happens even if it drives us all extinct.

Well, this instinct doesn't seem very compatible with safety, right? You're

seeing more people from the AI world coming out making noises about the need for for safety and how quickly this tech is evolving. Um, if this is your

is evolving. Um, if this is your perspective on the role of the tech and its future and ours, safety doesn't even seem like a

secondary concern. In fact, it seems

secondary concern. In fact, it seems like a barrier.

Yeah. So, there is some diversity of opinion among this group and I want to give them a fair shake. So, okay. Dan

Fella, who is the organizer of this worthy successor symposium, he actually says, "Look, what I care about is the flame of consciousness. You

know, it's like a really dumb thing to do.

building an unworthy bad AI that kills all life like you know just doing it in a really really unsafe way that leads to a catastrophe that ends up extinguishing

all conscious beings right on the on the planet and as perhaps in the universe like that would be really dumb. So he

says you know we shouldn't do it in an unnecessarily a straightforwardly dangerous way. So we

we should take some basic like safety precautions when we're making artificial general intelligence. Um but yeah,

general intelligence. Um but yeah, ultimately we should be okay with humanity getting attenuated and AI taking over.

Do they think intelligence and consciousness are the same thing?

I think there's a lot of squishiness around that. I think this community

around that. I think this community really tends to valorize intelligence a lot. Um

lot. Um they also tend to talk about it as if it's on a sort of a a single scale like runs along a single axis like there's just there's one kind of intelligence

rational intelligence you have more of it or you have less of it and if the AIs have more of it they're better than us.

Um when they talk about consciousness they tend to say again in a very sort of hierarchical way

look there are higher forms of consciousness. They'll say like look we

consciousness. They'll say like look we have a higher form of consciousness than a horseshoe crab. It's sure it sure is great that we exist and not just horseshoe crabs because we have access

to higher forms of consciousness. Um,

and so if the AI can have access to higher forms of consciousness than us, we should be happy for that to come into existence even if we go extinct. So you,

you know, whether they're using the term intelligence or consciousness, you do see this emphasis on, well, there's higher forms and lower forms. So what do they think humanity is for?

Like do they think that humanity as a species was once useful, that we we had a cosmic role, but we have just outlived our utility? Do they think our fate is

our utility? Do they think our fate is to be um a stepping stone for more enlightened creatures?

Yeah. So, a lot of them do think we are a stepping stone. They would say that AIs are our rightful heirs. They are the next step in cosmic evolution. Um and

that like look, we have evolved enough intelligence to be able to make these AIs.

Now, we really should. Um an interesting kind of exemplar of this is Elon Musk.

Perhaps you've heard of him. Um,

I have.

So, he used to, you know, really stake out what he called like a prohuman position and and he was more of a transhumanist. That is someone who still

transhumanist. That is someone who still wants to keep some version of humanity going. Uh, you know, maybe by merging us

going. Uh, you know, maybe by merging us with AI. Um, and in more recent years,

with AI. Um, and in more recent years, he's come around to the view that we are maybe just what he calls a biological bootloadader for AI, meaning we're just

this like small little program. Our job

is to just boot up the AI and then that's it. We're done. We fade away.

that's it. We're done. We fade away.

What are the the sorts of political beliefs that flow out of these these worldviews, whether it's transhumanism or successionist or accelerationist or posthumanist or whatever. I mean, I have

to imagine that wanting to see humanity replaced by machine intelligence leads to some incredibly disturbing politics.

Yeah.

Politically, one thing that happens when you have this ideology is you're probably going to be just generally a lot less concerned about like the

everyday problems that regular people face because like essentially those are all just sort of disposable, right? Like

the individuals don't matter so much. It

doesn't matter if you have like a bunch of kids in poverty or people who don't have housing or or whatever. um your

goal is just to be that species that gets the next worthy successor into existence. Um, the other like really

existence. Um, the other like really important political implication to my mind is this can really easily lead to a

eugenicist view and I know eugenics is a big word but I actually do think it's apt here because when you start making the case that there is an optimal being

you know some beings are better more morally worthy than others there's like a better way to be a more optimized way to be what you end up doing is contracting our our concept of like what are the

legitimate modes of existence or like legitimate ways of being in the world.

You contract that instead of expanding it. You contract it and contract it and

it. You contract it and contract it and you start saying well like to get to that optimized state we need to strip away all the cognitive or physical

features that are perceived as holding us back. Um that really quickly can lead

us back. Um that really quickly can lead to eugenics. Um, and indeed I've had

to eugenics. Um, and indeed I've had some pretty disturbing conversations with AI successionists where they don't

shy away from that. Um, they are pretty content to say, look, if one, you know, group is more intelligent and another group is dumber,

the dumb people should just get squashed out and they should be good with that.

Is this really just religion?

Yeah, I think it is religion. I think

this is a religious movement. uh though

it's wearing a secular disguise. The

truth is this is actually a very old religious idea.

You can draw a straight line going from Christian thinkers in the middle ages to Elon Musk's day. In the Middle Ages, Christian thinkers started to believe

that, you know, we need to like get back to the perfection of Adam before his fall from grace in the Garden of Eden.

Well, how do we do that? You got to be more. You got to imitate God. God was a

more. You got to imitate God. God was a creator, a maker. So add so we should also be creators and makers. That means

using technology. They developed this idea that we can use tech to uh you know tech progress would be moral progress.

We can use tech to achieve redemption and return us to that sort of perfect state that we had before the fall from grace. That idea gets carried all the

grace. That idea gets carried all the way through the Renaissance through the enlightenment. Um, in the Enlightenment,

enlightenment. Um, in the Enlightenment, you know, thinkers sort of shy away from the the religious garb. So, they put it in more secular terms. Instead of saying

we should infinitely ascend toward God, they say we should have indefinite human progress. Um, you know, instead of

progress. Um, you know, instead of saying uh we can become like the angels, they say we are infinitely perfectable, right? They they sort of phrase

right? They they sort of phrase everything in in secular language but it's always the same idea that we are capable of and have a duty to just up up

maximum progress. There is a

maximum progress. There is a hierarchical view we should always be ascending. Um and there is some

ascending. Um and there is some objective kind of vision of the good that we need to maximally ascend toward.

It is such a fundamentally religious story. I mean, it's so conventionally

story. I mean, it's so conventionally religious, right? Like,

religious, right? Like, yeah, they have a story which you've laid out.

That story tells you what humanity is.

It tells you what our role in the universe is.

Yep.

It tells you what history is moving toward, what salvation looks like. But

salvation is not heaven. It is super intelligence. And the God is not God.

intelligence. And the God is not God.

It's intelligence itself. Right? And the

apocalypse is not the end of the world.

It's the end of humanity as like the cosmic protagonist. Like like come on

cosmic protagonist. Like like come on now. I mean that's that is a that is a

now. I mean that's that is a that is a deeply religious story.

It's deeply religious. And the thing is like why? Well because religion is

like why? Well because religion is incredibly appealing to human psychology, right? There there's a

psychology, right? There there's a reason for that. One of the like very striking uh things I keep noticing when I talk to the AI successionists is teological thinking. Right? This is the

teological thinking. Right? This is the idea that the universe has a taos. It

has an ultimate end or goal. This is

very common in religion, right? The

universe has some ultimate destiny.

Um, and it's a staple of AI successionist thinking.

It's very easy for us to sit here and be like, "Oh, this is dumb." You know, like this is just a religion and just a religion as if religion is sort of just

like silly. But of course like religion

like silly. But of course like religion has been so powerful for millennia because it appeals to something very deep in us. And I think that is sort of

like a this deep desire we humans have to feel like there is some meaningful narrative to the universe. And

especially we love a transcendence narrative, right? Some story about how

narrative, right? Some story about how like we're going to overcome suffering once and for all. We're going to enter into some utopia. And this this really is a utopian vision. Even if it ends up

with humans going extinct, it's still pitched as the universe in the best state it could possibly be.

Well, look, to be clear, I don't think this is dumb. I think it is dangerous because like, you know, they're not merely saying AI might surpass us. They

are treating the possibility that it will as morally meaningful. They talk as if the universe is trying to wake up through intelligence, and our job is to help it do that, even if it means self annihilation.

Yep.

That is dangerous. So, I don't think this is dumb or stupid. I think it is dangerous.

I would agree. Yeah, I think it's dangerous. Um, and I think it's it's

dangerous. Um, and I think it's it's more influential right now than than most people might realize.

You're a kind of philosopher of religion. I mean, you know, I think

religion. I mean, you know, I think people believe what they believe because it it satisfies some emotional or spiritual or intellectual need. What is

belief in AI successionism doing for the people who believe in it? I mean, is it what you just said that this having this sense of of some transcendent purpose and this just happens to be a story that

jives with their kind of modern tech, you know, secular educated minds.

Yeah, I think it gives you a real hero's journey. Um, Mark Andre in his

journey. Um, Mark Andre in his techno optimist manifesto said this explicitly. Um, it gives you a heroic

explicitly. Um, it gives you a heroic quest, right? And people love to feel

quest, right? And people love to feel like they're in a Lord of the Rings story, right? They love to feel like

story, right? They love to feel like they're on a hero's journey. And what

could be more heroic than helping the universe fill up with consciousness and wake up to itself and achieve its ultimate destiny, right? Like it's so

grand. And when I talk to these AI

grand. And when I talk to these AI successionists, like these are not people who just hate humans and are trying to be cartoon villains. Like they

are very earnest. They stay up late at night trying to come up with what they think is like a lofty moral spiritual vision. And I think the the real

vision. And I think the the real challenge is for anyone who finds that vision of theirs repulsive, we need to articulate a counterveilling positive vision. I think the problem is that our

vision. I think the problem is that our you know the the natural candidate to give us that alternative vision which is humanism, right? The view that like

humanism, right? The view that like actually humans don't need some god or AI to rescue us. We have the ability and responsibility to improve our own lot

through our own efforts. Classical

humanism has not actually done a good job of preparing us for the 21st century because we face this question that the successionists are right to point out

which is how do we know how we should and shouldn't use tech to evolve our species? Like we have tech now that

species? Like we have tech now that could direct our evolution on purpose, right? whether that's brain computer

right? whether that's brain computer interfaces or crisper gene editing tech.

So like how should we make those decisions about what changes to make or not make to ourselves? I don't think classical humanism has given us a good answer. And so I think we actually need

answer. And so I think we actually need a new humanism to give us a a better vision than what the AI successionists are selling.

You know, I've become more open to the religious experience as I've gotten older. I was

much more hostile um in in my earlier years. But even when I was hostile,

years. But even when I was hostile, I still believed people who were, you know, atheist or secular or whatever, needed to take way more seriously like

the God-shaped hole in people's lives, right? Like the religion, whether you

right? Like the religion, whether you think it was capital T or not, served a deeply important metaphysical purpose for people. And if that's gone, then

for people. And if that's gone, then something is going to replace it. And

you better be very careful about what that is. It's not Christianity. It It's

that is. It's not Christianity. It It's

could be some form of political totalitarianism that sees history as God. And all we got to do is just fill a

God. And all we got to do is just fill a few mass graves to get to paradise, right? I mean, that is a religious

right? I mean, that is a religious project.

And that's the thing about this, right?

And this is such a like bizarrely 21st not even 21st century like 2026 like version of this. And I it's it's not surprising at all. But maybe what is

surprising is how many people in important positions are really believe it. I mean that's startling.

it. I mean that's startling.

I I totally agree. I mean religious ideas never die. They just put on new clothes. and we're now seeing the new

clothes. and we're now seeing the new clothes of very old religious ideas. Um,

but yeah, I think that it's our job to to offer something better. Now,

support for the gray area comes from Fetch Pet Insurance. Pets add so much to our lives. Really too much to say in

our lives. Really too much to say in words. We've got a cat and a dog here at

words. We've got a cat and a dog here at House Illing and they are the best.

However, we have been slammed a few times out of nowhere with really, really big vet bills. And every single time it happens, it stings. But what are you going to do? You got to do what you got

to do. It's your pet, right? Here's what

to do. It's your pet, right? Here's what

surprises a lot of pet owners. According

to a study from a pet insurance company from a few years ago, every 6 seconds, a pet owner in the US gets hit with a vet bill over $1,000. Emergency vet bills can run into the thousands easily, and

they always seem to land on a Sunday night, right? Be prepared for the

night, right? Be prepared for the unexpected with Fetch pet insurance.

According to consumer advocate.org, Fetch is the most complete coverage for dogs and cats. You get paid back up to 90% of vet bills for accidents,

illnesses, diagnostics, and more. Every

vet in the US and Canada is in network, so you can keep the vet you already trust. Plus, with Fetch, you can build a

trust. Plus, with Fetch, you can build a plan that fits your budget. And once

approved, your claims are paid back in as little as two days. That's about as easy as it gets, people. Don't wait

until your pet has a health problem. Go

to fetchpet.com/save

and get your free custom quote. It takes

about 2 minutes. That's

fetchpet.com/save.

Fetch. Love longer.

Support for the gray area comes from MintMobile. When you hear a deal that's

MintMobile. When you hear a deal that's too good to be true, you usually wonder what's the catch. But sometimes the catch is that there is no catch. Like

Mint Mobile offering premium wireless for just $15 a month. That's it. No

catch, just a good deal. Plain and

simple. Catches be gone. Mint Mobile

took what's wrong with wireless and made it right with premium wireless for 15 bucks a month. You can even bring your current phone and your number. It's

incredibly easy to get started. You can

sign up online and get 3 months of premium wireless service for just 15 bucks a month. To get your new wireless plan for just 15 bucks a month, go to mintmobile.com/gray

mintmobile.com/gray area. That's mintmobile.com/gray

area. That's mintmobile.com/gray

area. Cut your wireless bill to 15 bucks a month at mintmobile.com/gray

area. That's it. There's no catch. $45

upfront payment required, equivalent to $15 a month. New customers on first 3month plan only. Speed slower above 40 GB on unlimited plan. Additional taxes,

fees, and restrictions apply. See Mint

Mobile for details.

But you know there is something so wildly pathological don't you think about a creature that evolves to a point where it can no

longer tolerate its own creaturliness right like don't you think that that's what this is also deep down like people they reject our bodies our mortality our vulnerabilities all of it and like this

is a way to escape that. Yeah, I do think that deep down for some of these people, what's driving this is a real anti-humanism, a real disgust with

humanity, um, one of the successionists I spoke to was saying that he's really open to AI just taking over because humanity sucks, right? He was saying to

me like, "We have wars in 2026. Can you

believe it? We still have wars. It's so

stupid right?"

He's not totally wrong about that. We

No, like we have we have a lot of things that we could be doing better, right? I

I I get that impulse. It's I have compassion for that impulse. Um but

there's something I think very perverse about then saying, "Okay, so you know what? Even though I am a biological

what? Even though I am a biological organism and therefore have a hardwired drive to survive, I'm going to just like ignore that and be perfectly happy to go

extinct." Um it because I I think I want

extinct." Um it because I I think I want to pave the way for some higher form of life.

I I think that's really perverse. I

think that if you are truly open to all forms of consciousness, as the AI successionists kind of claim to be, they claim to have this radical openness.

Then guess what? You have to be truly open to all forms of consciousness, including ours, right? If all life is precious, human life is also precious.

the people that you you were talking to, do you do you ask them did you ask any of them why they think intelligence is the most valuable thing in the universe? Do they

have an argument that justifies why that is the thing that we should you know the altar upon which we we should sacrifice everything? Why is intelligence the god?

everything? Why is intelligence the god?

Different people kind of come at this differently. Um

differently. Um I think what a lot of them will tell you if you ask directly is intelligence makes stuff happen right like intelligence is the ability to achieve complex I mean they define it very

narrowly often too narrowly in my opinion but they'll say intelligence is the ability to achieve complex goals in the world um so it makes stuff happen and because they think the universe has

a certain ultimate destiny well for example if you want the whole universe to fill up with consciousness and wake up then you need beings that are intelligent enough to make the spaceship

ships uh to be able to go settle the stars, colonize the whole universe, turn the whole universe essentially into one giant data center um so that it can

compute um and and you know compute information and in some sense be be awake. Um the

awake. Um the tacit thing that they won't come out and say is that in my opinion they're obsessed with intelligence because they

have embibed ideas from the enlightenment era where rationality rational intelligence is prized above all.

And this idea that like reason is what makes humans special and should decide our destiny like goes all the way back to Aristotle. It's like we're the

to Aristotle. It's like we're the rational animal. that's what makes us

rational animal. that's what makes us unique and so that's what decides our role in the universe. So it's a really old idea. Um and I think that it got

old idea. Um and I think that it got really magnified in the enlightenment.

Um and especially under utilitarianism.

Utilitarians tend to be really obsessed with rationality. Um this gets sort of

with rationality. Um this gets sort of like elevated to be the ultimate thing.

So it does not surprise me that they elevate it to be like you know really on a pedestal.

Well this is where the political theorist in me recoils right because intelligence is great. I'm not

anti-intelligence. Yeah. Um at least most of the time. I don't try to be. Um

but intelligence also gave us industrial killing and the atom bomb. Like

intelligence unguided or unmed by moral values of some kind is maybe the worst most destructive force in the universe or at least in our

known universe. Is that even something

known universe. Is that even something that that um concerns them? I mean,

because it it seems conspicuously amoral or or uninterested in values and e and ethics of that kind. It's just it's it's all about intelligence and and maximizing or optimizing for it at all

cost.

Yeah. This is so ironic and paradoxical because I think the way they would tell it, they would say, "We we care more than you care, Sean, about what's really good. But to

determine what's really good, you have to look from the point of view of the universe." This is like a classic

universe." This is like a classic utilitarian slogan.

Looking from the point of view of the universe means looking with rational disinterest, right? Not just like as a

disinterest, right? Not just like as a human with human cares and and uh prejudices right?

Um, so they'll say like, "No, Sean, like we're actually better able than you to see what is truly good and what really matters and we're trying to maximize that." The the flip side that I wish

that." The the flip side that I wish they would engage with more is I actually um this book is currently propping up my microphone while we tape

this episode. Dialectic of

this episode. Dialectic of Enlightenment, the uh Herkimer and Adorno book, right? These are um theorists who

were kind of coming around in the era of the Holocaust and saying how could like Germany, this very rational, intelligent uh you know group of people

perpetuate these Nazi atrocities, right?

And they have this whole critique about how instrumental rationality if you know you just like maximize that and let it run on it, you know, go go to its own devices, it can lead to really horrible

things because they're not tuned to like your version of what's ethical. They're

tuned to what they think is most good according to the point of view of the universe.

Not that book, but that question. How

could that happen? How could one of the most educated civilized countries in the world ever descend into that kind of barbarism?

That is the question that that led to me getting a PhD in political theory. Like

that to me that was the question I had to answer for myself.

How does politics become religion in the worst way? And this is how

worst way? And this is how support for this show comes from Shopify. Whenever you're taking on

Shopify. Whenever you're taking on something new, it's easy to focus on what could go wrong. That's especially

true when you're starting a business where so much feels uncertain. But

there's another possibility worth considering. What if it works? What if

considering. What if it works? What if

your idea really connects with people and grows into something real? Shopify

can help you take that chance. They're

the commerce platform powering millions of businesses worldwide and nearly 10% of all e-commerce in the US. From

established brands like Allirds and Converse to entrepreneurs just getting started, their design tools make it easy to build the exact online presence you're imagining with hundreds of readytouse templates to choose from. And

with built-in marketing features, you can create full email and social campaigns in just a few clicks. So you

can reach your customers wherever they are. It's time to turn those whatifs

are. It's time to turn those whatifs into with Shopify today. You can sign up for your $1 per month trial period at shopify.com/vox.

shopify.com/vox.

You can go to shopify.com/vox.

That's shopify.com/vox.

Support for the gray area comes from HIMS. ED is more common than many people realize, and in a lot of cases, it may be easier to address than you'd expect.

With HIMS, you can connect online with a licensed provider to explore personalized treatment options, all from the comfort and privacy of home. HIMS

offers access to personalized prescription ED treatment options if prescribed. And if treatment is right

prescribed. And if treatment is right for you, they provide a range of options from trusted generics to customized plans with some generics costing up to 95% less than brand name alternatives.

Everything is handled online, making it easy, easy, easy to get expert care and treatment tailored to your needs. With

licensed medical providers guiding your care, HIMS put your health and goals first, helping you find a treatment plan that works for you. To get simple online access to personalized, affordable care

for ED, hair loss, weight loss, and more, you can visit hys.com/gray

area. That's hymns.com/gray

area for your free online visit.

hymns.com/gray

area. Prescription required. See website

for details and important safety information. Selenil is the generic

information. Selenil is the generic version of Viagra. Viagra is a registered trademark of Beatatric Specialty LLC. HIMS is not affiliated

Specialty LLC. HIMS is not affiliated with or endorsed by Vatric.

Support for the show comes from Task Rabbit. For a lot of people, summer

Rabbit. For a lot of people, summer means more projects. So, if you find yourself with a growing to-do list, Task Rabbit can help you keep from falling behind. Task Rabbit connects you with

behind. Task Rabbit connects you with skilled taskers in your area for moving, furniture assembly, home repairs, yard work, mounting, and more. You can search for a tasker based on cost, skill set,

availability, and past client reviews, so you know exactly who's showing up and can have confidence that they know what they are doing. And if it's short notice, no worries, because taskers are often available to help the same day.

Taskers have assembled over 3.4 million pieces of furniture, completed 700,000 home repairs, and handled 1.5 million moves and counting. So, our laundry room

was a complete objective disaster.

And we had to get a couple of giant cabinets. And what can I say? Your boy

cabinets. And what can I say? Your boy

just did not want to break out the toolbox and spend an entire afternoon putting all that stuff together. So, we

had a handyman come and do it. And it

was fantastic. When life happens, your to-do list grows. Get ahead of it now and get $15 off your first task at taskrabbit.com or on the TaskRabbit app

using promo code gray area. Taskers book

up fast, especially for same day tasks.

So, book trusted home help today. That's

$15 off your first task using promo code gray area with TaskRabbit app or taskrabbit.com.

taskrabbit.com.

Let me at least do minimally the devil's advocate thing for a second. I I will do it for for one

question, right? Um, if if AI really can

question, right? Um, if if AI really can create more intelligence, and I have no doubt about that, um, if it can create

more flourishing, certainly that's possible. If it could do more of that

possible. If it could do more of that than humans can, then why isn't preserving humanity just selfish speciesism? That's a word,

selfish speciesism? That's a word, right? Speciesism.

right? Speciesism.

Okay. Okay. So, yeah. What say you?

Good. You missed. I'm going to even turn the volume up on that question and you know say what what I've I people have said to me which is like not just

intelligent but if what if seal what if the AIs could be better on every dimension that you value about humanity so more intelligent but also more

spiritual more creative more kind more more all of that everything that we value about humanity what if they could do all that but just be better right

to which I say what what is it better for whom? What does this mean? This is

for whom? What does this mean? This is

assuming that there is some objective cosmic scoreboard hung over the universe. Um where everything is sort of

universe. Um where everything is sort of like ranked against some objective good or concept of moral value. Um and so whatever creature can best instantiate

that value wins and we should let it take over. Um, I don't think there is

take over. Um, I don't think there is such an objective cosmic scoreboard hanging over the universe.

I think that it does not make sense to speak of moral value or morality separate from a certain kind of moral agent, a certain

species. Our morality is a kind of

species. Our morality is a kind of morality that you have when you're a species that is like mamalian, so social, intelligent, has a lot of excess cognitive energy, right? like ability to

form complex social networks etc etc right for that particular species there's a particular kind of morality and humans do that um and if we one day

get conscious AIs they will do their own thing right but one is not better than the other because there is no like objective ranking in the universe that that is like there is one moral value to

rule them all um that I just don't believe that's a thing I think value is always value to someone and it doesn't matter to me if we are the most valuable

species for the grand unfolding of consciousness. We are valuable to us.

consciousness. We are valuable to us.

That's the thing, right? Like I'm not I've typically not been a fan of isms of any kind, but I am a proud speciesist because and this is maybe like the rub

for me. This is where like this this is

for me. This is where like this this is where I diverge like fundamentally from these people. Um like you just said a minute ago, they look out into the the

stars and they see um some kind of cosmic purpose and that and that cosmic purpose is like the unfolding of of reason and intelligence, right? And and

our role in that story is to serve it.

I look out into the universe and I see indifference, right? I see a I see a

right? I see a I see a you're such a postmodern I perhaps but like I yeah maybe I mean but but like I see a universe that is very clearly indifferent to like human

vanities and human concerns, right? And

so therefore like all we have is each other, right?

what what are we to do here but to like build lives for ourselves um and and create structures and institutions that allow us to like flourish as much as possible because at some point the

lights will go out and the universe will suffer a heat death and it you know it will all come to nothing anyway, right?

But like the idea that like um there is some clear transcendent purpose and that that purpose has an a special interest in us and our role in that it just it it

I just it doesn't it doesn't register with me. Right. But it's because I think

with me. Right. But it's because I think the universe doesn't really care about humans. That's like the foundation of my

humans. That's like the foundation of my humanism. That's like we of course we

humanism. That's like we of course we what else are we to do but to like take care of ourselves and each other, right? And like so the idea that someone

right? And like so the idea that someone could look at that same situation and and reason to the opposite conclusion which is that well we don't really matter how how how selfish of you like I

don't again it's not about smart or not.

I'm sure many of these people are are are plenty smart and much smarter than I am or ever will be. This is just like a fundamental philosophical divide that like cannot be bridged. I think it it

really is a philosophical divide and I think like the the thing underpinning a lot of it is a lot of these people are utilitarians or significantly influenced

by utilitarianism.

Um and if you're a utilitarian, you might be of the view that look, we're supposed to maximize aggregate happiness or moral value in the

universe. And so like and and we think

universe. And so like and and we think there's some objective kind of view of what that is. And so if some other creature can maximize that better than we can, well some other species, who are

we to stand in the way of that, right?

Like you you can see how they get to that from within their logical framework. But I'm with you. To me, that

framework. But I'm with you. To me, that whole logical framework just does not hold water. um you know, for all the the

hold water. um you know, for all the the reasons I was mentioning, um I don't think there's that sort of like objective cosmic scoreboard. Um I'm not

a utilitarian. Um

a utilitarian. Um but I I see how people end up there and I think it's really important for us to sort of tease out the the sort of steps

of how people get there so that we can kind of offer a different path.

So what is the response to this? Like so

you've I think I have to thrown this word humanism around several times here, right? And something you say in your

right? And something you say in your piece is that, you know, it's not enough, right, to give this humanist rejoiner and say, well, you know, the AI should serve us because of course it should, right?

Like, so if that's not enough, and you're right, it's probably not enough.

Clearly, it's not enough.

Um, what is, right? I'm not asking you to like give me your your philosophy of humanism in 90 seconds, but I can.

Oh, [ __ ] yeah. Then do it. Let's do it.

I'm here. I'm here for you, Sean.

This is why you're here. Um I think the old humanism has a lot of flaws that we need to update. So the teological thinking where we think the universe has

an ultimate destiny and we can figure out what it is. No, let's just admit we don't know the universe's ultimate destiny if if it even has one. So we

should keep a plurality of lifestyles possible. We should not create a society

possible. We should not create a society where everyone is you know technologically enhancing themselves um quote unquote optimizing themselves with tech. such that everyone else feels like

tech. such that everyone else feels like implicitly pressured to do so as well or else they'll fall behind. The language

of perfectability from enlightened humanism like that we should perfect the human. Those are those ideas are really

human. Those are those ideas are really dangerous because they contract the range of lifestyles that it's okay to live. We should instead adopt tech that

live. We should instead adopt tech that expands the range. Right? Like I'm not anti-tech. I'm perfectly happy to use

anti-tech. I'm perfectly happy to use tech to transform ourselves in ways that make more space for people to live more different kinds of lives. That's great.

The hierarchical view where, you know, you see this in antiquity, you see this in Renaissance humanism that species are on a hierarchy. We're it's 2026. We can

really move beyond that. Um, we can instead embrace, you know, sometimes scientists call this the diverse intelligences view, which is just saying, look, every species has its own

brand of smarts.

Each species is like amazingly adapted to its own environment and needs. They

have their own kind of intelligence.

Beautiful, precious, great. Like, we

don't need the hierarchy. And we don't need to try to look from the point of view of the universe. I'm sorry to all the utilitarians listening. I know they will hate me right now, but it is totally appropriate to look from the

point of view of humans. We are human moral agents. Our humanness is

moral agents. Our humanness is inextricable from our moral agency. It's

totally fine to look from the perspective of humans while just acknowledging that we're one out of many, many species that matter. I think

if we were to implement those four changes to the old humanism, we'd be well on our way to offering a more compelling positive vision than what the AI successionists are selling.

There's some interesting lines to toggle here, right? I mean, I we probably agree

here, right? I mean, I we probably agree that humanity is not fixed. Y

right. I mean, this is this is, you know, Nichch's distinction between being and becoming, right? Every living thing is like always in the process of becoming becoming something different

than it was before. That is evolution, right? And nobody knows what that is.

right? And nobody knows what that is.

So how does this humanism how do we take this this dual stand where we say humanity isn't fixed. Humanity is not the most necessarily the most important

thing in the universe. But can we say both of those things and then also um avoid some of these traps of saying well if we aren't fixed and we aren't the most important thing in the universe

then why does this really matter? Why

does our why does our ultimate fate really matter if this is the next phase of evolution matters to us. It matters to us.

Yeah.

That it has to be a good enough answer. It

just has to be because that I think that's it. I got

that's it. I got for me it really is because what's the alternative? The alternative is that you

alternative? The alternative is that you have to believe that there is some objective story of like some objective cosmic ranking of mattering and that we

humans are somehow able to like intuit it that the universe's cosmic scoreboard. Um

scoreboard. Um that just seems like a ginormous leap to me. Um but you you can see why it's

me. Um but you you can see why it's appealing, right? It gives people

appealing, right? It gives people it gives people the feeling that they are more enlightened than we are because

they can say our allegiance is to life itself. And that sounds like really

itself. And that sounds like really enlightened position um at first glance, but the more I talked to these people,

the more I thought I actually do not want my allegiance to be to life itself in some abstract sense. I don't even necessarily think that's possible. I am

a human through and through. And so I think it is totally appropriate for me to care about humans even as I also care about other species. But I don't think I

have some like moral obligation to bring a new species into existence um to maximize some like moral value um for

the universe that is objectively better than my moral value because I don't think there is some objectively better moral value. I just don't think that's a

moral value. I just don't think that's a thing. I think maybe also on some level

thing. I think maybe also on some level people are just looking at the world and saying this is happening whether it should or we want it to or

not. And so

not. And so they're in search maybe subconsciously for a story that they can tell themselves that enobles

that those conditions, right? That makes

it okay, turns out this isn't so bad.

Turns out us being replaced by AI is actually a good thing. And in fact, not only is it good, I can actually be a particip I can be a co-c collaborator.

I feel like Enobles is actually the perfect word here because Gon Verdon who is the founder of effective accelerationism or EA um which

is sort of in this constellation of groups um associated with AI successionism. He actually came out and

successionism. He actually came out and said explicitly, you know, why did he create EA? Well, because, you know, he

create EA? Well, because, you know, he he works in like sort of the the tech world and he was looking around at at a bunch of his sort of fellow tech colleagues and noticing that they were

all feeling like kind of crappy about themselves, as if they're like the villain maybe in humanity story and maybe they shouldn't be driving AI forward at an insane pace, etc. and they

were feeling like they're a bad character and Verdon says like he wanted to create an ideology that might make them feel like actually maybe they're they're a good character.

So you see there very nakedly the sort of psychological impulse to to sort of like rehab their own image for themselves.

All right. So last question I'll ask basically you know if um if one of these I keep saying guys but I have a sneaking suspicion this is mostly guys. Is that about right?

guys. Is that about right?

It's 98% guys.

Yeah. All right. Well, we we'll leave that there. But I Yeah, that's that's

that there. But I Yeah, that's that's what I assumed. Um if one of them said, "Okay, Sagal, I'll bite." All right. So

intelligence isn't God. Um so

intelligence isn't the utility function for humanity. All right. So what what

for humanity. All right. So what what what is what should we aim to maximize if it's not intelligence? What's your

answer?

I would say no one can tell you there here is the one answer and you should all do act based on my answer because the whole

point of what I'm saying is there is no one objective answer what we need is pluralism we need to create space for me to live out my values you to live out

your values right this is like basic liberalism and you know you should have some self-determination up until point where that's going to harm me, right?

Then we have to kind of temper each person's self-determination so we don't end up hurting each other too much. But

basically, we have to leave this pluralistic and open. And that is really hard for people. The poet John Keats had this amazing phrase negative capability

which was which he described as sort of like the ability to just stand in the open space of I don't know and not irritably and anxiously like grasp for

any answer to fill that abyss.

we need to have we need to really practice negative capability um and to just say I don't know all the ultimate answers of the universe and so the best

we can do is keep open a space um for a plurality of different kinds of life to flourish and like let's try to help each other flourish right like we can there's

a whole lot we can do to just raise the floor for every kind of being on earth right now to thrive more I can't say it any better I mean the show It's called the gray area for a

reason. I mean, honestly, maybe one of

reason. I mean, honestly, maybe one of the cardinal themes of the show for the entire time I've been doing it is living with and being comfortable with

uncertainty and not grasping at the expense of reality and the people around you for absolutes. Because what ends up

absolutes. Because what ends up happening is once you give people a story that is the story, that story collides with reality that something has to give. Yeah.

Right. And usually

usually they distort reality to preserve it and break anything that interrupts that. Um and that's often what happens,

that. Um and that's often what happens, right? So I mean, you know, whether it's

right? So I mean, you know, whether it's religion or politics or whatever it is, right? Like I there's there are a few

right? Like I there's there are a few things more dangerous than someone who really truly believes they have a hold of the capital T ultimate truth because

then everything everything that undercuts that becomes a threat.

Yep.

Including people.

Yeah.

And bad [ __ ] happens.

It's really hard to stay in the I don't know. I I wish us all loads of practice

know. I I wish us all loads of practice with this very hard skill. Hard for all of us, myself included. Um but that that's my wish for us to all all

practice that.

Amen sister. All right. Uh tell people the name of your newsletter and where people can go to to read it. It is

excellent and people should do that.

So in addition to reporting on AI, I write a newsletter called your mileage may vary. It's a philosophical advice

may vary. It's a philosophical advice column that responds to any and all of your ethical dilemmas. And you can find that by subscribing to Vox's Future Perfect newsletter.

All right, get on it, people. Saul, this

is great. We We should definitely do it again um on the other side of your leave. So, thanks for coming.

leave. So, thanks for coming.

Awesome. Thanks, Sean. Super fun.

Thanks for watching. Each week, we will be in your audio and video feeds with great interviews and a philosophy-minded look at tech, culture, politics, and

much more. Episodes of The Gray Area

much more. Episodes of The Gray Area drop on Mondays and Fridays on YouTube, Apple Podcast, Spotify, or wherever you get your podcasts. As always, we want to

know what you thought of the episode.

So, drop us a line at the gray area atvox.com. And if you enjoy our

atvox.com. And if you enjoy our reporting and want to hear more from Voxjournalist, then consider becoming a member of the Vox community on Patreon

at patreon.com/vox.

at patreon.com/vox.

Thanks for listening.