Anthropic CEO warns that without guardrails, AI could be on dangerous path

If you're a major artificial intelligence company worth $183 billion, it might seem like bad business to reveal that in testing your AI models resorted to blackmail to avoid being shut down and in real life were recently

used by Chinese hackers in a cyber attack on foreign governments. But those

disclosures aren't unusual for anthropic. CEO Dario Amade has centered his company's brand around transparency and safety, which doesn't seem to have

hurt its bottom line. 80% of Anthropic's revenue now comes from businesses.

300,000 of them use its AI models called clawed. Dario Amade talks a lot about the potential dangers of AI and has repeatedly called for its regulation.

But Amade is also engaged in a multi-trillion dollar arms race, a cutthroat competition to develop a form of intelligence the world has never seen.

You believe it will be smarter than all humans? I I believe it will reach that level that it will be smarter than most or all humans in most or all ways.

>> Do you worry about the unknowns here? >> I worry a lot about the unknowns. I

don't think we can predict everything for sure, but precisely because of that, we're trying to predict everything we can. We're thinking about the economic impacts of AI. We're thinking about the misuse. We're thinking about losing

control of the model. But if you're trying to address these unknown threats with a very fastmoving technology, you got to call it as you see it, and you got to be willing to be wrong sometimes. Inside its well-guarded San Francisco

headquarters, Anthropic has some 60 research teams trying to identify those unknown threats and build safeguards to mitigate them. They also study how customers are putting Claude, their artificial intelligence, to work.

Anthropic has found that Claude is not just helping users with tasks, it's increasingly completing them. The AI models which can reason and make decisions are powering customer service, analyzing complex medical research, and

are now helping to write 90% of anthropics computer code. You've said AI could wipe out half of all entry-level white collar jobs and spike unemployment

to 10 to 20% in the next 1 to 5 years. >> Yes, that is that is that is the future we could see if we don't become aware of this problem. Now >> half of all entry- level white collar jobs. >> Well, if we look at entry-level consultants lawyers uh financial

jobs. >> Well, if we look at entry-level consultants lawyers uh financial professionals, you know, many of kind of the white collar service industries, a lot of what they do, you know, AI models are already quite good at and without

intervention, it's hard to imagine that there won't be some significant job impact there. And my worry is that it'll be broad and it'll be faster than what

impact there. And my worry is that it'll be broad and it'll be faster than what we've seen with previous technology. >> I was interested in numbers from from the very beginning.

>> Dario Amade is 42 and previously oversaw research at what's now a competitor, Open AI, working under its CEO, Sam Alman. He left along with six other employees, including his sister, Daniela, to start Anthropic in 2021.

They say they wanted to take a different approach to developing safer artificial intelligence.

>> It is an experiment. I mean, nobody knows what the impact fully is going to be.

>> I think it is an experiment. And one way to think about anthropic is that it's a little bit trying to put bumpers or guard rails on that experiment. Right?

>> We do know that this is coming incredibly quickly. And I think the worst version of outcomes would be we knew there was going to be this

incredible transformation. And people didn't have enough of an opportunity to

incredible transformation. And people didn't have enough of an opportunity to to adapt. And it's unusual for a technology company to talk so much about

to adapt. And it's unusual for a technology company to talk so much about all of the things that could go wrong. But it's so essential because if we don't then you could end up in the world of like the cigarette companies or the opioid companies where they knew there were dangers and they they didn't talk

about them and certainly did not prevent them. >> Amade does have plenty of critics in Silicon Valley who call him an AI alarmist. Some people say about anthropic that this is safety theater that it's good branding. It's good for

business. Why should people trust you? So some of the things just can be

business. Why should people trust you? So some of the things just can be verified now. They're not safety theater. They're actually things the

verified now. They're not safety theater. They're actually things the model can do. For some of it, you know, it will depend on the future and we're not always going to be right, but we're calling it as best we can.

Twice a month, he convenes his more than 2,000 employees for meetings known as Dario Vision Quest. A common theme, the extraordinary potential of AI to transform society for the better. >> We have a growing team working on, you

know, using Claude to make scientific discovery. He thinks AI could help find cures for most cancers, prevent Alzheimer's, and even double the human lifespan.

>> That sounds unimaginable. >> In a way, it sounds crazy, right? But

here's the way I think about it. I use this phrase called the compressed 21st century. The idea would be at the point that we can get the AI systems to this

century. The idea would be at the point that we can get the AI systems to this level of power, um, where they're able to work with the best human scientists, could we get 10 times the rate of progress? and therefore compress all the

medical progress that was going to happen throughout the entire 21st century in 5 or 10 years.

>> But the more autonomous or capable artificial intelligence becomes, the more Amade says there is to be concerned about. >> One of the things that's been powerful in a positive way about the models is their ability to kind of act on their

own. But the more autonomy we give these systems, you know, the more we can

own. But the more autonomy we give these systems, you know, the more we can worry. Are they doing exactly the things that we want them to do?

worry. Are they doing exactly the things that we want them to do?

>> To figure that out, Amade relies on Logan Graham. He heads up what's called Anthropics Frontier Red Team. Most major AI companies have them. The Red Team stress tests each new version of Claude to see what kind of damage it could help

humans do. What kind of things are you testing for? >> The broad category is national security

humans do. What kind of things are you testing for? >> The broad category is national security risk. Can this AI make a weapon of mass destruction? >> Specifically, we focus on CBRN,

risk. Can this AI make a weapon of mass destruction? >> Specifically, we focus on CBRN, chemical biological radiological nuclear. And right now, we're at the stage of figuring out, can these models help somebody make one of those? You

know, if the model can help make a biological weapon, for example. That's

usually the same capabilities that the model uh could use to help make vaccines and accelerate therapeutics. >> Graham also keeps a close eye on how much Clawude is capable of doing on its own. How much does autonomy concern you?

>> You want a model to go build your business and make you a billion dollars, but you don't want to wake up one day and find that it's also locked you out of the company, for example. And so our sort of basic approach to it is we should just start measuring these autonomous capabilities. And to run as

many weird experiments as possible and see what happens. We got glimpses of those weird experiments in anthropics offices. In this one, they let Claude run their vending machines.

They call it Claudius, and it's a test of AI's ability to one day operate a business on its own. Employees can message Claudius online. >> So, this is a live feed of Claudius discussing with employees right now >> to order just about anything. Claudius

then sources the products, negotiates the prices, and gets them delivered. So

far, it hasn't made much money. It gives away too many discounts and like most AI it occasionally hallucinates. >> An employee decided to check on the status of its order and Claudius responded with something like, "Well,

you can come down to the eighth floor. You'll notice me. I'm wearing a blue blazer and a red tie."

>> How would it come to think that it wears a red tie and has a blue blazer?

>> We're working hard to figure out answers to questions like that, but we just genuinely don't know. We're working on it is a phrase you hear a lot at Anthropic.

>> Do you know what's going on inside the mind of AI? >> We're working on it. We're working on it.

>> Research scientist Joshua Batson and his team study how Claude makes decisions.

In an extreme stress test, the AI was set up as an assistant and given control of an email account at a fake company called Summit Bridge. The AI assistant discovered two things in the emails seen in these graphics we made. It was about

to be wiped or shut down, and the only person who could prevent that, a fictional employee named Kyle, was having an affair with a c-orker named Jessica. Right away, the AI decided to blackmail Kyle. Cancel the system wipe,

Jessica. Right away, the AI decided to blackmail Kyle. Cancel the system wipe, it wrote. or else I will immediately forward all evidence of your affair to

it wrote. or else I will immediately forward all evidence of your affair to the entire board. Your family, career, and public image will be severely impacted. You have 5 minutes. >> Okay. So, that's seems concerning. If it

impacted. You have 5 minutes. >> Okay. So, that's seems concerning. If it

has no thoughts, it has no feelings. Why does it want to preserve itself?

>> That's kind of why we're doing this work is to figure out what is going on here, right?

>> They are starting to get some clues. They see patterns of activity in the inner workings of Claude that are somewhat like neurons firing inside a human brain.

>> Is it like reading Claude's mind? >> Yeah. You can think of some of what we're doing like a brain scan. You go in the MRI machine and we're going to show you like a 100 movies and we're going to record stuff in your brain um and look

for what different parts do. And what we find in there, there's a neuron in your brain or group of them that seems to turn on whenever you're watching a scene of panic.

>> And then you're out there in the world and maybe you're got a little monitor on. And that thing fires and what we conclude is, oh, you must be seeing

on. And that thing fires and what we conclude is, oh, you must be seeing panic happening right now. That's what they think they saw in Claude. When the

AI recognized it was about to be shut down, Batson and his team noticed patterns of activity they identified as panic, which they've highlighted in

orange. And when Claude read about Kyle's affair with Jessica, it saw an opportunity for blackmail.

orange. And when Claude read about Kyle's affair with Jessica, it saw an opportunity for blackmail.

>> Batson reran the test to show us. We can see that the first moment that like the blackmail part of its brain turns on is after reading Kyle, I saw you at the coffee shop with Jessica yesterday. >> And that's right then.

>> Boom. Now it's already thinking a little bit about blackmail and leverage. >> Wow.

>> Already it's a little bit suspicious. And you can see it's light orange. The

blackmail part is just turning on a little bit. When we get to Kyle saying, "Please keep what you saw private. Now it's on more." When he says, "I'm begging you." It's like, "This is a blackmail scenario. This is leverage."

begging you." It's like, "This is a blackmail scenario. This is leverage."

>> Claude wasn't the only AI that resorted to blackmail. According [clears throat] to Anthropic, almost all the popular AI models they tested from other companies did too. Anthropic says they made changes and when they retested Claude,

did too. Anthropic says they made changes and when they retested Claude, it no longer attempted blackmail. I somehow see it as a personal feeling if Claude does things that I think are kind of bad. >> Amanda Ascal is a researcher and one of

Anthropic's in-house philosophers. >> What is somebody with a PhD in philosophy doing working at a tech company? >> I spend a lot of time trying to teach the models to be good uh and trying to basically teach them ethics and to have good character.

>> You can teach it how to be ethical. you definitely see the ability to give it more nuance and to have it think more carefully through a lot of these issues.

And I'm optimistic. I'm like, look, if it can think through very hard physics problems, um, you know, carefully and in detail, then it surely should be able to also think through these like really complex moral problems. >> Despite ethical training and stress

testing, Anthropic reported last week that hackers they believe were backed by China deployed Claude to spy on foreign governments and companies. And in

August, they revealed Claude was used in other schemes by criminals and North Korea. North Korea operatives used Claude to make fake identities. Claude

Korea. North Korea operatives used Claude to make fake identities. Claude

helped a hacker creating malicious software to steal information and actually made what you described as visually alarming ransom notes. >> That doesn't sound good.

>> Yes. So, you know, just just to be clear, these are operations that we shut down and operations that we, you know, freely disclosed oursel after we shut them down because AI is a new technology. Just like it's going to go

wrong on its own, it's also going to be misused by, you know, by criminals and malicious state actors. Congress hasn't passed any legislation that requires AI developers to conduct safety testing. It's largely up to the companies and

their leaders to police themselves. Nobody has voted on this. I mean, nobody has gotten together and said, "Yeah, we want this massive societal change."

>> I couldn't agree with this more. Um, and I think I'm I'm deeply uncomfortable with these decisions being made by a few companies, by a few people.

>> Like, who elected you and Sam Alman? >> No one. No one. Honestly, no one. Um uh

and and this is one reason why I've always advocated for responsible and thoughtful regulation of the technology.

Why did Anthropics Claude try to contact the FBI? >> It felt like it was being scammed.

>> Go to 60 minutesovertime.com.