Mark Zuckerberg on Llama 3.1, Open Source, AI Agents, Safety, and more

the first time we're releasing a 405 billion parameter model by far the most sophisticated open source model that that I think anyone has has put out I was a little shocked by how directly you called out apple and their closed

approach can you kind of expand on that and where Apple has been a blocker for meta yeah I think it's a little bit Soul crushing when you go build features that are what you believe is good for for

your community and then you're told that you can't ship them because some company wants to put you in a box so that they they can better compete with you can talk more about your long-term vision

for what you think is going to happen with AI and possibly AGI in the future all right Mark thanks so much for doing this um obviously huge AI

announcement from meta today can you give us the rundown on everything released and why it's important yeah sure so I mean the big release today first of all happy to be doing this big

fan of what you do um the big release today is llama 3.1 and we're releasing three models um the first time we're releasing a 405 billion parameter model um so it's by

far the most sophisticated open source model that that I think anyone is has put out um and and it really kind of is competitive with some of the leading closed models and in some areas is even

ahead so I'm really excited to see what people do with that especially now that we're making it so that our the community policies around llama allow people to use it as a teacher model to

distill and fine-tune and um and basically create whatever other models they want with it um in addition to that we've distilled the 405 billion

parameter model down to make newer and updated and now leading um for for their size 70 billion and 8 billion parameter models they also have really good

performance um really good kind of cost per performance ratios so I'm really excited to see what everyone does with this um you know I mean taking a step back I think this is a pretty big moment

for open source AI um you know I've been reflecting on this and I I kind of think it's you know I thought for a while that open source AI was going to become the industry

standard and I thought that it would basically follow the path that Linux did where you know if you if you um just go back to before Linux was popular there

you know there were all these companies that had their own closed versions of Unix and at the time you know there's nothing that was sort of that sophisticated that had ever been done as an open source project and people

thought hey no this is like the closed model of development is the only way to do something that's this Advanced and at first Linux kind of got its foot hold because um it was cheaper because

developers could customize it in different ways and then over time as the ecosystem built out it you know got more scrutiny so it actually became the more secure one it became the more advanced one um there were more partners that

basically built more capabilities in the case of Linux more drivers um and things like that that basically ended up making it have more capabilities as well than any closed source unic so I think that

this moment with llama 3.1 is kind of like that inflection point where um I think llama has the the opportunity to become the open- source AI standard for

open source to become the standard the industry standard for for AI and even in the places where it's not yet ahead on performance it leads on on kind of cost

and on on customizability and on the ability to take the model and find tune it and do all the things that you want with it um so I think that those are just huge advantages that that we're going to see developers take and we're

focusing on building out this partner ecosystem and there are going to be all these different capabilities that get built out around it so yeah really excited to talk about all that yeah I've seen all the benchmarks it looks

incredible um obviously the first ever open source Frontier Model uh for 405b um are there any specific real world use cases that you're really excited about

to see people build with the model well the thing that I'm most excited about is seeing people use it to distill and and fine-tune their own models right it's I

mean like you're saying I me this is the first open source Frontier level model but it's not the first Frontier level model so there have been other models that sort of have that capacity and yeah people are going to want to do inference

directly on the 405 because it's you know by our our estimates it's going to be about 50% cheaper I think than than GPT 40 um to do that directly and um so

I think that that that obviously makes a difference to a lot of people but the thing that I think is really new in the world with this is the because it's open

weights um the ability to take the model and distill it down to whatever size that you want to use it for synthetic data generation to use it as a teacher

model um you know so our vision for the future it's not just okay it was never that there's going to be One Singular thing I think this is like open AI sort of as this Vision that they're going to build kind of one big AI anthropic does

too Google does to it's never been our vision our our vision is that there should be lots of different models I think every startup out there every Enterprise governments they all kind of

want to have their own custom models and and yeah when the closed ecosystem was so much better than open source it was just better to take the vanilla Clos thing off the shelf because even though

you could customize open source there was still some gap between the performance that you could get but now we don't see that anymore right now as open source basically closes the Gap I think you're just going to see this wide

proliferation of models where people now have the incentive to basically customize and build and train exactly the right size model for what they're doing um train their data into it they're going to have the tools to do it

because of a lot of the partner Integrations that the companies like um like Amazon are doing with AWS or data bricks um or different folks like that who are building these whole Suites of

services for distilling and fine-tuning um open models so I think that that's going to be the thing that's new here and that's really exciting is how far can that get pushed and um and that's a completely new capability in the world

because there hasn't been an open source or open weight model of of of kind of this um sophistication that's ever been released before yeah it's a really big

deal and um how are you educating developers to use these tools and more broadly does meta have like a plan or strategy to educate the rest of the

world on open source and why it really matters yeah so I'd say before llama 3.1 our approach I mean the reason that meta fundamentally is investing in this is we basically want to know that we have

access to to a leading model um you know because of some of our our history of of kind of how mobile worked and things like that um we didn't want to be in a position where we had to rely on some

competitor for this kind of fundamental technology so we built it for ourselves and before llama 3.1 you know we we kind of add this Instinct that if we made it

open source there would be a community that would grow around it and that would actually extend the capabilities and make it more valuable for everyone including us um because at the end of the day this isn't just a technology it's an ecosystem right that that that

you're developing so um in order for this to end up being a useful thing for us there also needs to be a broad ecosystem one of the big changes that that I think we see with llama 3.1 is instead of just building it for

ourselves and throwing it over the wall and letting developers use it this time we're really taking a much more proactive stance on building Partnerships and making sure that um

there's this whole ecosystem of companies that can do interesting things with the model and conserve developers in ways that we're not going to right we're not a public cloud provider right

it's Ian we're not AWS or or Google or Azure right it's so developers aren't going to come to us to build their stuff but we want to make sure that all those public clouds are well equipped to do this um that goes for some of the more

basic functionality like just hosting the models and serving inference um but we also want to make it some of the new stuff that's going to be possible with

this like distillation and fine-tuning um that there's specific services that are set up for that and you know a lot of those Services haven't been that well built out before because that's not really a thing that you could do as well with closed models so um there was

specific work that we had to do with those Partners to enable that at the same time I think that there are also going to be folks um like grock right who are doing really interesting work on

really kind of ultra low latency um inference and I'm really excited to get this in their hands and they they're building something for launch that basically is um is gonna is is is going

to enable that too and then there's this whole set of Enterprise companies so folks like like Dell or you know scale.ai or deoe or Accenture who um work with all the Enterprises around the

world on on technology deployments and I think those are going to be a lot of the folks who um help build kind of custom models for you know whether they're large Enterprises or governments a lot

of folks who basically want to have a model that's their model that they can train their custom data into it but they um you know a lot of companies don't want to send their data over an API to you know Google or open AI it's and not

because um not because those companies have any particular privacy issue but it's the same reason why people like encryption and WhatsApp right they people just want kind of secure by

Design architectures where their data can can kind of stay with them and um I think that there's G to be a whole market around things like that that get built out too so I'm pretty excited about all of this but yeah we're taking

a much more proactive position this time and helping to build out the ecosystem because I think that's how how this um grows and just becomes more valuable for everyone yeah I love how close you are

to the developer Community um just me myself being in the community I know for a fact that people really want these private and local models so um moving on

to your letter uh so alongside of the meta announcements you published a letter and um the first portion really focused on why open source is good for developers and I felt like really spot

on um can you talk more about like what the broad societal implications are of Open Source AI yeah I mean my view is

that open source is a really important ingredient to having a positive AI future and there are all these awesome things that AI is going to bring um in terms of productivity gains and

creativity enhancements for people and hopefully it'll help us with research and things like that but I think open source is an important part of how we make sure that this benefits everyone

and is accessible to everyone and isn't something that's just locked into a handful of big companies um at the same time I actually think that open source

is going to end up being the safer and more secure way to develop AI I know that there's sort of a debate today about is open are safe and I actually take the different position on it not

only do I think it's safe I think it's safer than the alternative of clo development and you know I sort of break it down into you know there there are lots of different kinds of harm so it's you can't just talk about one type of

thing but um on this I think that there's there's unintentional harms so the system goes off the rails in some way um that people didn't intend and then there's

intentional harms where you have like some Bad actors trying to use the system to do something bad when it comes to unintentional harms which I think by the way it's worth noting that like most of the Sci-Fi scenarios that people worry

about of AI just going rogue um are kind of unintentional I I actually think that open source should be safer on that because it's it will have more scrutiny

it'll have more transparency um and I I think all the developers who use it with all the Lama guard and the safety tools that that it comes with um there's going to be so much scrutiny and testing and

pressure on those that my guess is that it will have kind of just like traditional open source software um any kind of issues with it I think will be ironed out and fixed a lot quicker than

with when the closed models so I think you got you've got that on on kind of unintentional harm which is why I think most of of the discussion around safety for open source revolves around intentional harm it's okay it's open

it's out there how are you going to stop Bad actors from from doing it doing bad things with it um there I think you basically want to probably divide the

problem into kind of smaller actors like an individual or or um or some kind of smaller group that's trying to create some some some Mayhem and larger actors

who are more sophisticated have huge amounts of resources like big nation states I think it's kind of a different mix for the two of those um you know for

the smaller actors and my view on this is that um you know the way that we've that I think that having a balance of power on this is super important um you know what we've done in managing our

social networks is we have all these kind of Bad actors who are trying to do kind of bad stuff on our networks and the way and a lot of times they deploy AI systems to do that and the way that we stop them and identify them is by

having more sophisticated AI systems that have more compute to go find what they're doing so I think that this is actually pretty similar to that governments and law enforcement essentially maintain order in society

it's like yeah you have a bunch of Rogue people who might be committing crimes but you know generally the police forces and the militaries are much better funded have more resources and I think that that's basically going to be true

here as a matter of fact I think what you want is for open source to be widely deployed which I think that there's sort of a risk if it's closed that that's not the case but when it's open you're G to have all these big institutions that

have a ton of resources that they can basically deploy these systems in a way that I think will check Bad actors then you get to um the question of of

basically you know folks like China like large sophisticated actors and one of the questions that you sometimes he debated is like okay if you're open sourcing the really Advanced models how

do you make it so that that it doesn't get to to to China or they're not going to use that against us and um and that's sometimes an argument that people have for hey you should lock down development but I think that that's sort of missing

a few things one is that in order for this all to work the US has to have an advantage in the first place or or the the west and and in kind of our advantage is basically open and

decentralized Innovation it's not just a small number of big companies or Labs it's startups and universities and individuals hacking on things who are't even parts of companies and that's a big part of it and you don't want to shut

that down so and I think if you do you you you increase the chance that we don't even lead in the first place but then I think you get to the the issue which is okay like China or not even

China any government um you know there all the risks of of kind of stealing the models and and Espionage I mean a lot of the models fit on you know a hard drive that you can you know quickly put in

your backpack or whatever and it's um I I just think we need to be realistic about How likely it is that we can secure um and not not just not us but like any of the tech companies can

secure any of these um models long-term against very sophisticated efforts to do that so my own fear is that if we lock down development we end up in a world

where basically you have a small number of companies Plus all the adversaries who can steal the model are the only ones who have access but all the startups all the universities all the individual hackers are kind of just left

out and and don't have the ability to do this so my own view is that a realistic aim that we should hope for is um is that we use open source to basically

develop the leading and most robust ecosystem in the world in that we have an expectation that our companies work closely with our government and Allied

governments on National Security so that way our governments can persistently just be integrating the latest technology and have a you know whatever it is a six-month Advantage eight-month advantage on our adversaries and I think

that that's you I don't know that that in this world you get a 10-year permanent Advantage but I think a a kind of Perpetual lead actually will make us more safe um in one where we're leading than the model that others are

advocating which is okay you have a small number of closed labs they lock down development we probably risk being in the lead at all like probably the other governments are are are are getting access to it it's that that's my

view I I actually think on on both these things spreading prosperity for for um more evenly around the world making it that there can be more progress and on safety I think we're basically just

going to find over time that open source leads um look there going to be issues right it's like we'll have to mitigate the issues we're going to test everything rigorously we do we work with governments on all the stuff we'll continue doing that um but that's my

view of of kind of where the equilibrium I think will settle out given what I know today yeah let's talk about more on the benefits of Open Source AI um so another thing you mentioned in your letter is

that open source a can accelerate Innovation and economic growth um how is this already happening and how do you see this happening more in the future yeah I

mean I I think that there's a version of this which AI will do no matter how it's developed um and then there's a version of this that I think benefits from open

source specifically so I think that AI has more potential than any other single technology that's being developed right now to increase productivity accelerate

the economy um make it that kind of every person has the ability to be more creative and and and produce more interesting things and I think that that's all going to be great I I also think I I hope that it'll help out with

science and um medical research and and things like that um there are a lot of folks today though who don't necessarily have access to the

ability to fine-tune or build their own state-of-the-art models so they're sort of limited to what these large Labs do um and like I just said I I think um you

know one of the defining aspects of our culture around Innovation as a sort of a country or or Society is like it's not just big companies that do it right there's all these startups and hackers

and academics and people in University and I think you want to give all of those folks access to state-of-the-art models that they can build on top of not just that they can run which is what they have today with with the closed

vendors but that they can build on top of and and tweak and distill down to smaller models that they can run on their laptop or their phone or whatever other device they're building and I think that that's just going to unlock a

ton of progress there's also a version of this where there are you can look at it by um you know Nation to um you know so it's not just that startups might not

have the resources or universities might not have the resources to go train their own um you know large scale Foundation models now or in the future but um but there are lot of countries that aren't going to have the ability to to do that

because I mean you know pretty soon these things are going to cost many billions of dollars to train and um I think that having the ability for different countries and entrepreneurs

and different countries and businesses to use it to serve people better and and just do better work is going to be something that that basically like lifts

all boats around the world and um just has a massive kind of equalizing effect so I think that that's really positive and you know that's one of the places where we get the most um thanks for this

is not actually The Tech Community but it's just it's like different developing countries there other countries that want to have access to the technology and do stuff with it um but but wouldn't necessarily have the the tech field

themselves to produce something that is state-of-the-art that their businesses can build on top of but once they have it it's actually pretty easy to go train your own thing so um so that's uh that's that's a pretty neat part of this yeah I

love it um one other thing I want to touch on from the letter is I was a little shocked by how directly you called out apple and their closed

approach um can you kind of expand on that and where Apple has been a blocker for meta and potentially others yeah I mean my point in there is

more it's a little more philosophical on how it affected my own kind of approach towards things and um and psychologically sort of affected how I

think about building stuff um I actually don't know how they're going to approach AI um you know they do some open development they do some closed development um you know by the way I think it's worth noting like I don't actually consider myself to be an open

source Zealot I just think that in this case um I I think that open models are going to be the standard and I think that that's going to be good for the world but we do open development we do close development so I get it right and and I'm not saying that Apple's necessarily going to be in the wrong

place on this um for AI but if you look back over the last 10 or 15 years um it has been a formative experience

for us is building our services on top of platforms that are controlled by our competitors and for a number of different

incentives they they absolutely from my perspective apply different rules to kind of limit what we can do and and yeah they have all these taxes and you know at some point we we did um we've

done some analysis that we we think we'd be you know way more profitable um if it weren't for some of these arbitrary rules and and I think a lot of other businesses would be too but you know

honestly the the money part I think um it's annoying but for me it's not the biggest thing it's you I think it's a little bit Soul crushing when you go build features that are that you that

are what you believe is good for for your community and then you're told that you can't ship them because some company wants to put you in a box so that they they can better compete with you and my

concern for AI at this point isn't actually Apple it's more the other companies and how that would evolve and I I think to some degree it's not even that I'm not even saying that they're

like bad people it's it's um I think that there's just a physics and incentive structure to the system where you know if you build a closed system then eventually there are all these

forces on you that that sort of kind of push you to to to kind of clamp down on things and um I I I I think that it will

be a healthier ecosystem if it's developed more like the web but um but more capable and I think that you know because of how mobile developed

were the closed model one right I it's like apple I think has has really reaped most of the benefits um in terms of you know they there might be more Android phones out there but like apple gets like almost all the profits of for

mobile phones I I think there's a bit of recency by bias because these are these are long Cycles right I mean the the iPhone I think it came out in 2007 right so we're almost 20 years into this thing

it's a long cycle um but it's easy to forget the fact that the Clos model doesn't always win um if you go back to PCS um you know I know a lot of people

have especially if you're using the Linux analogy people don't necessarily consider Windows to be maximally open but compared to the um the Apple approach of of kind of coupling your

operating system with um with the device the windows approach was a more open ecosystem and it won and part of my hope for the next generation of platforms

which includes both Ai and the work that we're doing an augmented in virtual reality is to you know meta wants to be on the side of building the open ecosystems and it's not just that we want to build something that's an

alternative to the closed ecosystem I want to restore the industry to the state where the open ecosystem is is actually the one that is leading um so I think it's possible I I think we'll you

know I think we're making good steps on that in both Ai and AR and VR U but but that's something that I just personally and philosophically care about given the

the kind of limits on on creativity that I I've felt have sort of been applied to to our industry by the closed model of of mobile development over the last 105 years I want to go deeper on that point

that you just mentioned of restoring the industry to the state where the open ecosystem is the one that's leading um and so obviously right now we

have llama 3.1 405b just came out it's competitive and it even beats some of the best closed models across key benchmarks which is insane in its own right uh but in your letter you also

mentioned that llama 4 is expected to become the most advanced model in the industry are there any specific things about llama 4 that you're really excited

about oh man I mean it's um you know we're just doing 3.1 for for for llama now I I think it might be a little early to to talk about llama 4 but um but

we've got the compute cluster set up um we've got a bunch of the data set up we we kind of have a sense of what what the the architecture is going to be and and and I've run a bunch of research experiments to to kind of Max that out

so I I do think that um llama 4 is going to be another big leap on top of llama 3 I think we have um a bunch more progress that we can make I mean this is the first dot release for llama

um there's more that I'd like to do um including launching the uh the the multimodal models um which we we kind of had an unfortunate setback on on on that

um but but I think we're going to be launching them probably everywhere outside of the EU um uh hopefully over the next few months but um yeah probably

a little early to talk about llama 4 but but it is going to be awesome and it has been one of the interesting things in running the company is basically planning out the compute clusters and

data trajectories for not just llama 4 but you know the next um probably four or five versions of llama because I mean

these are long lead Time Investments to build out these data centers and and the power around them and um the chip architectures and the networking architecture so all this stuff um so

yeah I I realize that's that's a bit of a non-answer for now other than just some general excitement but um I don't know let's uh I I I think llama 3

deserves at least um you know llama 3.1 deserves at least a week of of kind of just processing um you know what we've put out there before we get into talking about the future that's fair and I

completely agree that the next couple weeks are going to be wild just with 3.1 uh but it's still super exciting obviously to hear that meta has already get anything ready for llama 4 um on

that front though can you talk more about your long-term vision for what you think is going to happen with AI and possibly AGI in the future yeah I mean I'm happy to talk about it both from a

technical perspective and a product perspective but since we've mostly talked about the models so far maybe I'll start with um with the products so our vision is that there should be a lot

of different AIS out there and AI services not just kind of One Singular AI um and that really informs the open source approach it's you know it also

informs the product road map so yeah we we have meta AI um met AI is doing quite well my goal was for it to be the most used AI assistant in the world by the end of the year I think we're well on

track for that we'll probably hit it hit that Milestone um you know few months before the end of the year um and and look on that you know I think we just

have the ability in the business model to basically build in the most advanced models in the world and offer it to everyone for free so I think that that's

a you know kind of a huge Advantage um it's really easy to use from all of our apps um so I'm pretty excited about how that's going so that's yeah we have the basic assistant um and and I think that

that's going to be a big deal but even more than that a lot of what we're focused on is giving every Creator and every small business um the ability to

create AI agents for themselves um making it so that every person on our platforms can create their own AI agents that they want to interact with and if you think about it these are just huge

spaces right so there are hundreds of millions of small businesses in the world and one of the things I think is really important is basically making it so with a relatively small amount of

work um a business can basically you know few Taps um stand up an AI agent for themselves that uh can do customer support sales communicate with all their

people all their customers I kind of think that every business in the the future just like they have an email address and a website and a social media presence today I think every business is going to have a um an AI agent that

their customers can talk to in the future and we want to enable that for all of those that's that's going to be hundreds of millions maybe billions of what kind of small business agents

similar deal for creators um there are more than it's more than 200 million people on our platforms who consider themselves creators who basically use our platform um in a way that is

primarily for you know building a Community um you know put putting out content feel like it's it's kind of like a part of their job is is doing that and they all have this basic issue which is that there aren't enough hours in the

day to engage with their Community as much as they'd like and likewise I think that their communities would generally want more of their time but um but again not enough hours in the day so I just

think it's a there's going to be a huge unlock where basically every Creator can pull in all their information from social media can train these systems um to reflect their values and their business objectives and what they're

trying to do and then people can can interact with that it'll be almost like this almost artistic artifact that creators create that um that people can can can kind of interact with in

different ways and then and that's not even getting into all the different ways that I think people are going to be able to create you know different AI agents for themselves to do different things so I think we're going to live in a world where there are going to be hundreds of

millions of billions of different AI agents eventually probably more AI agents than there are people in the world and um and the people just interact with them in all these different ways so that's part of you

know that's the product Vision um obviously there's a lot of business opportunity in that that's where we want to go make money so we don't want to we're not going to make money from selling access to the model itself um because again we're not a public Cloud

company we will make money by building the best products an important ingredient to the best products is building is having the best models which having the best kind of ecosystem around

open source will help us do so that's why it's kind of all aligned for us and why this is is going to end up I think being really valuable for us to build the the highest quality products that we

can um and and have the best business results by by kind of building out this open source Community but but it's also why it's all philosophically aligned right we don't we just don't believe that there's going to be kind of one big

AI whether it's a product or a model that everyone uses we kind of fundamentally believe in having this broad diversity and different set of models and that you know every business

and um you people are just going to want a lot of their own stuff they're going to make and I think that's kind of going to be interesting it's going to be a lot of what makes this interesting yeah I think it's really interesting to see how met is integrating the tech directly

into his products and obviously giving a frontier level AI model for free to billions of users is a huge deal um but on that I have a final question around

skepticism and for context in the 1990s skepticism around the internet was everywhere but eventually it became almost irrational to oppose it

and it kind of feels like we're in a similar trajectory with skepticism around AI right now do you think we're kind of at that early stage and there'll

be a point where anti-a sentiment will be viewed similar to the internet today and what kind of factors you think will be crucial in changing that perspective

I think there are different ways of uh different ways that people can be worried about something I mean it's um I mean financially one thing that I'm

quite aware of is the internet um had a big bubble burst before it succeeded and you know so all the people who were very long on the internet um were eventually

right but sometimes things take a little longer to develop than you think and you just need to have the commitment to see that through and um that's something that I'm aware of because yeah I mean I I I'm really excited about you know all the unlocks that we're going to get from

llama 3 and then llama 4 and then llama 5 and I think that's going to translate into better products but realistically um it's hard to know in advance when something is good enough that you're

going to have a product that billions of people use and then when it's ready to to kind of be a large business and I mean look we're all spending you know a lot of capital and and on basically training these models so I I think that

people are going to be probably losing money for quite a while um but but I don't know maybe maybe that'll all happen quicker I'm it's it's it's hard to know exactly um the other part of this that I think you are more getting

at is people's concern about what it means for their livelihoods and on that this is one of the reasons why I think the open source approach the approach of um lots of different models out there

that are kind of personalized and customized to to every business and every Creator and every person um I think that's important because if this

of develops in a way where it's just a you know a small number of companies that build the products and benefit and people use the products and maybe they like talking to to you know an AI

assistant and that's valuable for them but you know if that if this doesn't in some way help lift all boats then I think you end up eventually getting a

backlash and part of what I've spent some time thinking about after just looking at how the kind of Web 2.0 stuff

developed is in the next generation of Technologies around AI around AR and VR how do we create not just a kind of

thriving set of products um and and kind of economic kind of productivity gains but how do we have like a better and more sustainable political economy around it where there's just way more

people who are who who feel like they're they're kind of bought in or benefiting from this um and supportive of of of the system and you know I I I thought we did that reasonably well with social media

but um but I you know just looking at some of the feedback and some of the response from from the world um I think that it's going to be important to do that even better with AI and some of the new technologies in order to to uh

mitigate some of the concerns that people are going to have about what this is going to mean for their livelihoods and and and jobs and their lives yeah I don't think anyone could have said it any better um but Mark it's been amazing

speaking with you thanks so much for doing this and thanks for everything you and met are doing for the AI Community happy to do it and um I'm really looking forward to seeing what people build