YouTube Video

All right, folks, we're going to have some fun in this session.

My name is Apoorva.

I'm going to be your instructor for the next nine weeks or so.

Here's what we're going to go through today.

I'm going to talk a little bit about myself.

Why do I do this.

Some logistics on what to expect.

Quiz.

Yep, I'm that guy.

We're going to have quiz on day one.

And the biggest question that we've all been wrestling with, where's the money?

Where's the money in AI?

Some of you know me, but my journey started in India.

I moved to Singapore.

I met a couple of Singaporean folks here earlier.

I started my career at Palantir with Sunil and a couple of other folks about 13 years ago, 14 years ago.

Led a variety of engineering teams. All that to say, we wrote a lot of spark in government buildings.

And I came back to Stanford for grad school, which is when I got tired of writing Spark in government buildings.

And now, I lead Altimeter.

I don't know how many of you have heard of Altimeter, but Altimeter is an investment firm.

We focus on fairly concentrated form of investing.

We've got two businesses.

We've got a public business and a private business.

And then I got the biggest promotion of my life six months ago.

I'm now the proud dad.

This is, as people have told me, the biggest investment I will make.

Some have called it the one with the most guaranteed negative IRR.

I think it's the most guaranteed positive IRR, not financial, but that's me.

I live across the street.

Reach out with questions.

I want to be here as a resource to you guys.

I'm joined by an incredible TA in Chloe Feng.

Reach out to her if you have harder questions-- [APPLAUSE] - And make the most of it.

We've got a great session lined up for you guys.

The course is designed to be no more than three hours a week, and that includes class readings all the time you spend arguing with ChatGPT, with Claude about whether it can do your assignment for you.

So it's about an hour of class.

It's about an hour or two of readings.

And basically, the format is we're going to do guest speakers every single class, next class onwards.

Chatham House Rules a lot of guest speakers will share, maybe overshare.

So please don't record what they're saying.

We'll have an optional dinner with some of them right afterwards.

You guys are welcome to join.

Chloe will arrange the logistics.

Grading is easy.

50/50, show up to class.

If Ali Ghodsi can show up to class, you can show up to class.

And the other half is an assignment that we will release at the end of the course.

Yeah, it's conversational.

Ask questions, be involved.

The more you're involved, the you're going to get out of it.

And honestly, what's in it for me is I'm going to learn the most from you guys.

This is the course schedule over the next nine weeks.

Lots of great speakers, one more impressive than the other.

As for me, honestly, putting this calendar together has been my job for the last couple of weeks and months.

Chloe knows all about it.

But be present.

These are all incredible leaders running incredible businesses across the stack from Semis to Infra to energy on the infrastructure side to models.

You're going to have folks from OpenAI and Anthropic and a bunch of applications and agents.

So ask all your hardest questions.

Save them for the speakers.

They're going to love it.

And we're going to assign some readings.

So why should you take this course?

What should you achieve in this course?

What is a good thing to get out of this?

You know, honestly, I thought about this, and I was just telling one of the students here of how it all began.

I come back to campus once a year and I talk about all that's happening.

And typically, it's in the context of finding great people.

I realized that this is such a big supercycle.

We know it.

We believe it.

At Altimeter, we have positioned our entire focus around it.

And I did not find a course that goes deep in a way that I would have liked to be when I was an undergrad here or a grad student here.

And I thought about in five years, everybody was going to ask you, hey, did you see it coming?

You were at the start of it.

You were around when ChatGPT was launched.

You were around when the tectonic plates were forming and clay was forming.

And I think you want to be able to say yes.

So half of you are going to start an AI company and the other half are going to fund it.

[CHUCKLING] So at least you should know where to spend the Series A money that you're going to raise.

And at the very minimum, you'll have a sense of what not to go, or at least have mental models for, hey, this business that I'm looking at or considering starting or considering funding or considering joining, what are the right questions to be asking?

What are the laws of physics that govern this business at this part of the cycle?

I think it's going to be the biggest one yet.

And I'm excited that you guys are here to study alongside us.

I'm going to spend some time on this slide because this is the punchline.

How many of you have seen a version of this before?

Well, those of you who did the readings, thank you.

I appreciate it.

We did include a notebook LM for those who were more auditory inclined.

But let's talk about this for a second.

What is going on here is actually probably the biggest question in generative AI right now, which is, if you listen to any of the earnings calls from the hyperscalers or even NVIDIA and others,

is we are investing so much into the CapEx.

We're investing so much into building these data centers.

It's a 5-layer cake, as Jensen calls it.

Energy chips power, interconnects memory all that to give you a data center that you can either rent by the hour or by the token, that you can go train models on and serve those models.

And then the question is, hey, these models that you built, are they creating economic value?

That is basically the right-hand side of this chart.

And to make an analog of the biggest technology revolutions that I have seen, internet 25 years ago, mobile 20 years ago, cloud, probably the most recent one, 10 years ago.

I put up one of those charts on cloud, but on the readings, you'll see the same for internet and mobile and cloud.

And that's the shape of the cloud ecosystem.

The cloud ecosystem looks dramatically different than the AI ecosystem.

Anybody have guesses as to why that's the case or your theory on why it's so different, or reasons why this might look like-- we're not going to call it a pyramid, we're going to call it a triangle, inverted triangle.

Go ahead.

Is it because it's still early in the upside for AI?

Yeah definitely.

[INAUDIBLE] Definitely early.

That's a good guess, yeah.

Any others?

Any other thoughts?

Or maybe because NVIDIA has a monopoly, so they can charge whatever.

Can you ask that question again next week when we have the folks from NVIDIA?

But no, good, it's a great point.

They do have a stranglehold, right?

One of the charts we had in the readings was the market share that NVIDIA has on all of the compute right now, and it's up there.

Any other thoughts or hypotheses on why this is so different?

Yeah, I don't know, is it the cloud sectors seem to be able to leverage the hardware to generate [INAUDIBLE] and AI haven't really got there yet.

We know how software ate the world.

As Marc Andreessen said, software ate the world because I could build software, you could build software, and I could distribute it to millions of people.

And the marginal cost of running that software was close to 0.

These software businesses ran at 80% some even at 90% gross margins.

That is not the case with this new economic model of AI, because if we have a set of users using Cursor or using-- you hear all these stories about large bit scale businesses that are still not profitable at billions of dollars of revenue scale, is because of that, is because the incremental user

of an AI application is not free.

It's not marginally free.

It's actually quite a bit more expensive to have AI users, because it turns out you've got to burn those GPUs.

And I would say everything you guys said from it being early to NVIDIA being dominant, we'll call it, to the physics of the problem are very different of how inference is run is certainly where we are right now.

So I think that's the case right now.

I might add another dimension to it, which I spoke about in the readings was, we analyzed what happened in internet.

We analyzed what happened in mobile and cloud, and how many years did it take for these triangles to flip?

And one of the examples we take is AWS.

AWS started in the year 2004.

AWS has its first customer in Netflix in 2010, and ultimately Amazon shifted fully to AWS in 2012.

Eight years from breaking ground, eight years from first CapEx investment cycle.

I don't know if any of you were around reading earnings reports 20 years ago, but the big debate was, hey, is Amazon going to go bankrupt?

And that was the biggest question everybody had about the build out of AWS.

And thankfully, nobody, at least yet, is on the verge of bankruptcy.

But these are large numbers.

So we'll come back to this slide.

But I would say this is the central theme of the course that we're going to explore.

We're going to have speakers from some of the companies that are listed here to others.

And the central theme that we're going to pick around is like, hey, in your field, with the NVIDIA speakers, are you a dominant force?

How long are you going to stay to be the dominant force?

What are the forces that you're most worried about?

Who are the ASICs that you're most worried about?

What are the pricing compression vectors for your business?

To the folks at Anthropic and OpenAI, who we're going to talk a lot about profitability, is you're serving a billion-user franchise at OpenAI.

With the Anthropic folks, honestly, 100% of this class is on Claude.

So we'll ask them about, is this group of users profitable?

How do you think about profitability?

Is ads going to be a bigger source of revenue than subscriptions?

And then for the folks in the middle, which is the inference layer, this is the most competitive part of the whole ecosystem.

There's a lot of startups that are doing really well.

They're winning so far.

But you've also got the hyperscalers who want to have a dominant say in that layer.

So honestly, the jury's still out.

And the biggest question there is, are you a feature or a platform?

A lot of new businesses that we are seeing on the infrastructure side that they feel very good ideas.

But if you ask yourself the question, hey, why is this not a part of AWS?

You are thinking about maybe it should be a part of AWS.

So for the speakers, we're going to talk a lot about that.

Any questions before we jump into the quiz?

Go ahead.

I'm curious how you think about-- so on the right-hand side, like the triangle being-- like the Application Layer being small, how do you think about including incumbent platforms into that?

Maybe like Salesforce.

Maybe [INAUDIBLE] revenue.

Like, would you-- do you include them as part of that-- in line with that pyramid and how it shifts over time?

It's a great question and I might add, Salesforce, Palantir, there's a series of-- let's call them "old economy businesses" that are reinventing themselves to have skews of products that are, in the case of Salesforce, Einstein, in the case of Palantir, AIP.

In the case of-- there's a series of these.

And the answer is yes, there should be.

The answer is yes, that they should be.

The way I solve for that in this calculation is I get the model revenue.

And so if you were running Salesforce, you're probably running either one of the big models or running inference.

So their spend is captured in the app layer by way of the substrate.

It's very hard to extract that out from public disclosures.

But yeah, we should.

Yeah.

Is a large part of the bottom part of the right-hand side pyramid, basically, buying capacity for future revenue, which are the group at the top, which is what we're not seeing in the [INAUDIBLE].

It's a great question.

And maybe just to rephrase the question, the question is, hey, is there a timing mismatch in the build out of the Semis layer?

Because typically, you build Semis for a five-year period or a six-year period.

But the application revenue is for right now.

It's a great question.

And that's what makes the lower half of this, call it triangle, somewhat cyclical.

And you go through phases of CapEx cycles.

Think of it as like laying down the railroads.

That is very much the case.

And so there's a chart in the readings for what happened in the mobile super cycle.

Something very similar happened.

The first inning had inflated market caps for a lot of the CapEx heavy businesses.

And so if you think about a basket of CapEx names to call it steady state names, you should expect that.

And so I suspect we're so early that that's happening as well.

I'm curious to see how you think about Google, because I see that you labeled Google as Google Cloud there on the [INAUDIBLE].

But Google also have their own Gemini models [INAUDIBLE] TPUs, and they're perfectly [INAUDIBLE].

How many of you are kind of positioned in this triangle?

Any large conglomerate like Google deserves to be-- we have to call business units.

So I would put the TPU business unit in Semis.

We include that here as we counted the revenue.

Their GCP unit is in the Infrastructure Layer.

And then the Gemini unit is at the Apps Layer.

We have a chart later that we'll talk a little bit about.

Gemini is actually one of the most used consumer applications.

It's the second most used consumer application right now.

And the biggest question there is, how much of that is coming from the distribution advantage that Google has to it meritocratic being such a good application?

The jury's still out, but we'll get into it.

Yeah.

Well, let's-- go ahead.

I just have a question about prediction.

Right now, it looks like this triangle shape, if it were to be successful, perhaps it should become inverted.

But what does an unsuccessful new technology look like?

Does it stay a triangle?

How would you be able to predict whether this would be good or not?

Yeah, I'm not sure the-- maybe rephrasing your question, I might say it out like, what is the stable equilibrium of this industry?

I think it is pretty clear that AI is unlikely to be a fad.

Is unlikely to be an unsuccessful endeavor.

And I think about the stable equilibrium of this chart quite a bit.

In fact, I got into a little bit of a debate on Twitter with somebody quite smart and who's thought a lot about this exact question of what is the stable equilibrium.

And my guess is that it might stay this way for longer than I anticipated.

In the cloud, I think that range is about a decade.

I have a feeling this might stay longer-- this way longer, because of just how hard it is to get the substrate right.

But there will be one or two unlocks.

I couldn't tell you what they are, but for example, if one of the ASIC programs at one of the hyperscalers, be it Google's TPU or Meta's MTIA or the folks at Amazon and OpenAI and Microsoft and all the labs that we don't even know about exist breakout success, I suspect that'll be the biggest repricing of that layer.

The other catalyst could be I think about the hyperscaler CapEx guidance in earnings calls, which, by the way, I recommend everybody here to listen to.

Four times a year, you'll have public company CEOs tell you their biggest questions, their biggest things that they're thinking about, and I recommend listening to those.

If they just stopped guiding to big numbers on CapEx, because that would imply that the current equilibrium does not work.

So that's why the second thing that could happen.

And so you see, there's a lot of news about the guidance that all the hyperscalers give about their CapEx for that reason.

Go ahead.

[INAUDIBLE] of training versus inference.

Because my sense is if-- the only way this flips is if inference is meaningfully larger than training.

And I'm curious to hear your thoughts on when you think that will happen, because then you're seeing that [INAUDIBLE] or you feel like they will stop spending on training because they're not seeing [INAUDIBLE]?

[INAUDIBLE] It's a great question, and it is probably one of the nuggets of information that NVIDIA's earnings calls, have the most sought after nugget of what is NVIDIA's share of inference in their fleet.

Last I checked, it was about 40%, or they quoted to be about 40%.

Meaning that if they were selling a million GPUs, assuming full utilization, about 40% of them were used for inference and the other 60% for training.

I suspect that number will increase over time in favor of inference, but I couldn't tell you when and how it'll happen, because there's a lot of training still going on in the world.

And the shape of the training workload, as you know, looks very different from the shape of the inference workload.

A training workload is very predictable, high utilization for a short period of time.

The inference workload is very burst usage.

Typically when humans are awake until the agents take over, maybe then it'll be 24/7, and harder to predict.

It goes down around Christmas for some reason.

It goes down around Thanksgiving for some reason.

But I think that might be the case, though we, at least in this calculation, we try to capture it because it's a mix and-- but it's a good hypothesis.

[INAUDIBLE] where would probability be?

It's on slide 16.

We'll come to it.

We'll come to it.

I'll give you the answer.

The most profitable part of the stack is the Semis layer by a long shot.

NVIDIA's data center revenues or on a gross margin of-- About 75%.

Don't quote me on it.

It's like plus or minus a couple percentage points from there.

Whereas, I estimate some of the Application Layer revenues to be somewhere between-- depending on who you ask, between 0% and 30%.

And so the gap is quite wide.

And I think the reason to that-- I mean it's a theory that a gentleman here had is, there's one player who runs the tables on the Semis here.

And so it's very much the case.

And in fact, if you looked at this from a profitability perspective, it's even more concentrated.

The triangle is even more concentrated.

I'll flash that in a second when we get to it but it's a great question.

Go ahead.

Do you think [INAUDIBLE]?

Yeah, that is definitely a big part of it, is that we've gone through the investment cycle in cloud.

It's definitely an element to it.

Go ahead.

If all these Infra companies like Google, AWS [INAUDIBLE] all their own TPUs, and media is also doing inference, OpenAI is also searching for some ASICs, where does all these ASICs inference startups want to sell to if they are using their own chip?

There's $300 billion of revenue to fight about.

But to answer your question, about half of that, as Jensen discloses on the earnings calls, is from the big hyperscalers.

So those are probably going to be your primary customers.

So if you were starting a chip company today, you would have a very-- the shape of your customer base is a very small number of very large orders.

It's a very different shape from building a consumer business or an enterprise software business.

And then you might have a long tail of other enterprises, though I wouldn't bank on it because I think they just go to the cloud providers.

If you were thinking about starting a chip company, it should be your number one consideration, is like, which of the five are you going to sell to first?

Last question.

[INAUDIBLE] do you get a small handful of winners in each of these layers?

It takes multiple years for that to play out.

Maybe I'm wrong, but I feel like in the past there hasn't been a fully vertically integrated layer [INAUDIBLE].

I get that Google is fully vertically integrated on the right-hand side, but wondering how that shift the balance of power this side.

What a great question.

The biggest winner on the internet super cycle is probably Google.

It's about $3 trillion in market cap, has near 99% market share in search.

I would say that that's a pretty vertically integrated player, right?

They run their own file server to search to ads on top to the user experience.

Let's see, the next one is Mobile.

The winner of that Super cycle is Apple.

What?

With $2.5 trillion or so in market cap.

You called it already.

The next one is, let's say social.

Meta is probably the big winner in social.

They're not as fully integrated.

And what is their market cap like? $2 trillion

or something right now?

Pretty dominant, but maybe they lost a trillion because they didn't fully go down to the servers.

And then the cloud is fairly heterogeneous.

We don't have a single player that won the cloud.

You've got the three big oligopolies in AWS, GCP and Azure, but they're not fully integrated.

And NVIDIA has been trying a lot.

NVIDIA has been trying-- I don't know if you've heard of DGX Cloud, which is their cloud effort to build the cloud ecosystem.

Obviously, they've got a series of vertical apps that they're trying.

So yeah, you might be on to something.

Folks, I know it's a Thursday evening at 5 o'clock, probably the last thing standing between the weekend and your weekend.

So I don't want to be that person.

So I'm going to jump into the part that wakes you up.

I do actually have-- this is a quiz that we're going to go through.

I'm going to give you a hint about the companies that we're going to go through.

I do have a prize for the winner.

This is the prize so you're motivated.

And you win points on two grounds.

One is by being right, and the other is by being fast.

The fastest way to be fast is to do fast inference and drop that thing into Claude.

Please, you're welcome to do that.

Just give the human players five seconds, let them go at it and let them win the analog way.

And if you really want to use Claude, you're welcome to do it, just give them five seconds.

All right, so this is question number one.

[CHATTER] Ready for the next?

The software engineers in the room might have an unfair advantage.

So I wanted to spend the next maybe 10 minutes or so going into some of the hypotheses that I have about what's going on and why the value is accruing in the manner that it is.

I think there was a question, a very good question, about profitability and how it gets magnified.

So we'll jump through that.

But again, feel free to stop me if you have any questions.

I have a feeling we're going to have very little time left, and I do want to end on time.

You guys remember this?

And I painted it slightly differently on the next chart, which is, I did the same exercise that I did that I posted about two years ago.

And what it looked like two years ago was this thing on the left, where the ecosystem was obviously a lot smaller.

It was about five times smaller.

Shockingly, the shape of it hasn't changed much.

This is despite heroic growth.

And if you look at the revenue that was added, about $350 billion or so of revenue added, a good, like, 75% of it just went straight to Semis in the last two years.

Despite apps having grown more than 10x, it still hasn't made that big of a dent.

And so I was like, OK, well, let's dig deeper into this.

If you started to open up each of these cells and you're like, hey, what companies make up each of those parts?

Most of that 300 is in NVIDIA, as you guys know.

the Apps is actually, two companies make up about 90% of it.

Anybody want to guess which those two are?

The Infra segment is the one that has the most competitive intensity, as we discussed.

It is probably the place where there's the biggest battle brewing both sideways, but also across the stack.

It's also the place that has the highest metabolic rate in that there's a lot of companies being formed, there's a lot of companies that are getting bought out.

And I would say it's the most competitive, but also the most unstable of the equilibriums that we have right now.

And the question that we think about as we think about investing, as you guys will think about investing your time, is how much time will this chart that has moved such little in the last two years, what is the amount of time it will take to get to cloud software like shape?

Is it five years?

Is it 10 years?

Is it 15 years?

Is it never?

Maybe it just stays that way.

We do think it will happen.

We think it will happen at some point, but it's not happening nearly fast enough.

The second thing that we've been thinking a lot about as we think about the future of AI is-- I don't know if you guys saw this chart in the readings, but consumer AI, which is the biggest, call it market for AI right now outside of coding, has incredibly high usage on-- ChatGPT, most of it is free.

About 95% of the users are free.

And Gemini, whose-- I don't know if you guys saw, but Demis, who leads DeepMind, announced that they were not planning to do ads as a subscription-- as a revenue model.

We've been thinking a lot about hey, how big do these businesses get?

What is the monetization engine of these businesses?

Do you think a subscription business will be larger or ads business will be larger?

And so what I did was I looked at the largest consumer franchises outside of AI.

And so you will see that-- I mean, you all know these products.

There's a class of products that have gotten to three billion users scale.

These are almost near mandatory products to live your lives.

This is WhatsApp and Chrome, which you could not live without.

Then there's a class of products that the 1,5 billion to 2 billion users scale, which are social, these are social products like Instagram, TikTok, and Facebook, they're not mandatory, but they're exhibit very good network effects.

If my friend Chloe is on one of these, I'm more likely to be there.

And then you've got the third category of mainstream consumer products that are neither mandatory, that are neither extremely social, but I would call niche products.

If you're shopping, you're going to Amazon.

If you're looking for music, you're going to Spotify.

If you're looking for a good debate, you're going on Twitter or cat videos.

Any guesses on where closer to which of these will ChatGPT and Gemini are right now?

And the answer is on the next slide, so we'll get it quickly.

Would you guess that ChatGPT or the leading AI application's terminal scale will be closer to a mandatory app like YouTube or WhatsApp, a social app like Instagram or TikTok, or a niche app like Spotify or Twitter?

Any guesses.

If not, I'll reveal the question.

Go ahead.

I would say on the YouTube, WhatsApp scale because it would be a daily utility.

Yeah.

People would just be using it daily as part of their normal life.

Yeah.

You are-- well, let me show you the answer and we'll come back to your biases.

Any other guesses?

Any different guesses?

Go ahead.

I think it's closer to Facebook time.

[INAUDIBLE] Yeah.

Yeah, that's right.

You're certainly right right now.

OK, I'll show you guys the answer.

This is how they fare if you plot them all together.

ChatGPT has just overtaken the niche app category.

Gemini still has not.

You were right that it's heading towards social.

Personally, I would have loved, as an investor at OpenAI, I would have loved for it to start heading towards the core utility.

But one of the biggest questions that we ask ourselves is, is knowledge work work that everybody does?

Is the work of-- ChatGPT is not a place where you're messaging other folks, yet it's not a place where you're getting your email inbox or your dopamine fix.

It's a place where you go and you have to do active work.

You have to go ask a question.

And the number of people in the world who are asking active questions of technology is not the entirety of the population that's online.

There's about 8 billion people on the planet.

4 billion of them are online.

The rough economics of consumer applications, Alphabet has about 4 billion users.

They monetize them at about $100 a user a year.

Meta's got about 3.5 billion users that monetize at about $70 a user a year.

The leading AI provider ChatGPT, has got about a billion users that are monetized at about $10 a user a year.

And so the question is, how do we get the billion up to 4 billion?

I'm not sure knowledge work is the answer.

I think we'd have to go beyond knowledge work.

And then the second question is, how do we get the $10 a user per year up from 10 to 100?

And I'm not sure subscription is the answer.

I suspect we'll have to go into ads.

And I suspect the ads that ChatGPT will be able to serve or Claude will be able to serve will have a lot better pricing because they will understand your intent, that you will be logged in, very good attribution, a lot more trust.

And I think that will be the other big headline this year.

And you heard it here first.

It'll be a big deal.

There's a lot of alpha in understanding the ad model really well.

Once again, 10 years ago, the Facebook IPO, there was a lot of short reports on Facebook because people said, hey, well, these ads worked on a computer, they're not going to work on a phone.

Why?

Because there's no space on a phone.

Shocker, we found the space on a phone.

The same thing's going on right now, which is, while I'm having this conversation, it is a very personal conversation, I don't want to be interrupted by advertisements.

That's the bigger debate.

I couldn't tell you what it's going to be like, but I am optimistic that we will find it.

And I think that's going to be a big unlock, a big unlock for this economic model.

And so we'll dig into that in one of the speaker sessions later this year.

I've got a bunch more slides.

We are at time.

Thank you.