How Notebook LM Makes Anything Interesting | Steven Johnson on AI & Creativity

I remember we had an early meeting and I was talking about this vision and Clay said, yes, but Steven, the thing you have to remember is that we are not normal. We can't design the product too much for that use case. It turned

not normal. We can't design the product too much for that use case. It turned

out that building a tool that in the very early days was really centered on the needs of one idiosyncratic user, me, ended up being kind of the fastest way to get to a tool that was actually useful for millions of people. He came

up with a great slogan for Notebook, which is, make anything interesting.

Second episode of AI to the DNA, and we have the hottest tool on the AI planet at the moment, which is Notebook LM. It's number 13 on the top 50 list the moment we record this video. We met the co -founder of Notebook LM, Steven B. Johnson. And the interesting part, he is not a scientist.

He's also not a machine learning expert. He is an author originally. And the story of how he came to Google and what he designed with the tool and with the team from Google DeepMind is really, really interesting. And if you're new to this podcast, my name is Christoph Magnusson. This is originally made for the YouTube channel. You'll

find the YouTube channel under my name, Christoph Magnusson, and all the episodes. But if

you're here on the podcast channel, subscribe and let us know if you have special episodes that you want to listen to. Great. Sitting here with Steven Johnson, the co -founder of Notebook LM. To me, at the moment, the hottest tool when it comes to using generative AI. And I say that in brackets and very carefully, because

there is more to this tool. Well, that's very nice of you to say. I'd

love to hear that. Thanks. It's great to be here. And Steven, we had a nice, nice chat already. And when you did a little bit of insight and presentation of how you use Notebook LM, we will share some of that. But this is a podcast and a video, obviously both. What was in the beginning of this

tool a moment that you realize, okay, that all I tried didn't work out. I'm stuck here. Yeah. I mean, we really, there were a lot of moments like that. You know, when you're trying to do something that's genuinely new, like you always kind of run into those points and part of the, I don't know, the essential toolkit as a founder is to just keep powering through those moments.

But we kind of made a very conscious decision in the early days of Notebook to, in a sense, design the software to be ahead of where the models were in terms of their capabilities. And so we built, you know, there's this core idea at the center of Notebook, which is source

grounding. You know, you're giving the model the documents, the project you're working on, and

grounding. You know, you're giving the model the documents, the project you're working on, and it becomes effectively an expert in those documents and answers questions faithfully based on those documents. And you should be able to put in 50 documents or 100 documents, sources

documents. And you should be able to put in 50 documents or 100 documents, sources as we call them. And the truth is, like, while we had that vision and when we started working on it in 2022 and early 2023, the models just weren't capable of doing that at any level. You know, Google's models

or anybody else's models, really. And so we kind of deliberately assumed that the underlying AI technology would progress to enable us to do this magical stuff that we had in our minds rather than trying to design the app for where the models were at that particular moment in time. And that meant

that, in some sense, the, like, product was kind of broken for the first year of internal testing with it. And then even in some of the period where we were kind of a public experiment, it just, it was good, but it wasn't nearly as good as we wanted it to be. And I remember, like, distinctly this moment in kind of the spring of 2024 when we had just switched over to

Gemini 1 .5 Pro. And that was the point at which it really started to

Gemini 1 .5 Pro. And that was the point at which it really started to sing a little bit. And I remember one of my colleagues, Simon, who now is one of the product leads at Notebook, but at the time was working on other things. And I saw him in the hallways in Mountain View, and he was like,

things. And I saw him in the hallways in Mountain View, and he was like, Steven, notebook finally works. And I was like, well, I mean, it kind of worked before, but yeah, I get your point. You know, it definitely was a point of like, okay, we've been waiting for this. And so I think that's a generally like a good principle to when you have an underlying foundational technology

that's improving at such rapid rates to really think about in your early stage designs, like where it's going to be in a year and design for that, because otherwise you, know, you won't be ambitious enough, I think. And here comes the, the, the part of the story that excites me the most. You need to be a

physicist, a deep research expert, a developer for that, or would you say it helps to be income from a different profession? Maybe you give some insight about your backstory.

It's kind of interesting. Yeah. New York Times article I remember especially. Yeah. So I,

I had this very unusual background, I think for, um, new products being developed at Google, uh, that, that came directly out of the philosophy of labs, which is the kind of new division, um, that notebook still, still belongs to, um, where we're, it was founded a couple of years ago, three or four years ago, um, to do

more cutting edge work with new technologies, largely AI, um, that were focused on actual applications rather than open -ended research.

Like Google has an amazing research division, but, um, this was a bit more like let's do edgy experimental things with an eye to like turning them into shipping products.

And so it complements a lot of what research does, but does it in a slightly different way. And one of the early ideas for labs was that, they would bring in people from other fields, um, who were not necessarily software developers, um, or designers, um, and have them in the, in the room where it happens as it were, like helping to build products. And in a sense,

I was the first guinea pig for that, for that agenda. Um,

and I had, you know, I spent most of my career as a writer, um, and journalist, I've written, you know, more than a dozen books on innovation and science and technology, but I had always been very interested in using technology, using software to help me write those books and to research those books. So, so

for instance, Google search was a huge, I'm old enough that like when Google search came along, it was a huge breakthrough for me as a, as a researcher. Google

Scholar was a huge breakthrough for me as a researcher. Um, but I also used a bunch of other tools like Devon Think, which I believe is actually a German, um, kind of knowledge management software, uh, and Scrivener for writing and things like that.

So I'd always like been an early adopter and kind of an evangelist for using software to help me think and, and come up with ideas and manage my research and stuff like that. And so the folks who had kind of founded labs, this guy, Clay Bevor, who since left and Josh Woodward, who now runs labs and also

runs the Gemini app, they had been reading my, my books and my articles over the years. And they read this article that I wrote in the New York times

the years. And they read this article that I wrote in the New York times magazine about language models in the spring of 2022. And before

chat GPT was very controversial. All he was saying was that language models are a really big deal and we should take them seriously and they have a lot of opportunity and people hated on it for weeks after that article came out. But, um,

Clay and Josh, read the article and apparently Josh turned to Clay one day and said, Hey, what if we brought Steven into labs, maybe part time?

He's been dreaming of this like ideal research software his whole career. He's

clearly obsessed with language models. Like maybe he could help us build something new. And

so they cold called me, you know, out of the blue, basically clay did and said, Hey, we've got a crazy idea. What would, what do you think about coming and taking a part -time position at Google? And we'll give you a couple of engineers and a designer and maybe we make something. And so I said, yes, that sounds great. And like three years later, here we are. On the, on the top

sounds great. And like three years later, here we are. On the, on the top 20 list actually of the hottest AI tools. I think we were last list. I

saw we were the 13th biggest AI tool, which is pretty crazy. And I, that was before the back to school, we have a lot of students. And so we've been growing a ton in the last month and a half with students coming back from vacation. So I wouldn't be surprised if we're higher on that list now, but

from vacation. So I wouldn't be surprised if we're higher on that list now, but anyway, we'll see what was the original starting point. If you take us back, because like, to me, it's all about turning, as I said, from a tool tourist into champions league of users being like really at the cutting edge. And you mentioned

that, that you still have it from the very past where you said, Hey, I was always an early adopter. So you always understood intuitively to leverage a tool, but it's a much different thing to build a tool than to just use it. Yeah.

Well, I would say one thing that I had, that I brought with me to Google that I had been kind of maintaining for 20 years as a, as a serious knowledge management nerd, is that I had been collecting quotes from books that I read as part of my research. And I started

doing this literally in the late nineties where I would actually like have to type up, like I would highlight a passage in a print book and then I would type up, or I would sometimes have a research assistant type them up. Cause I

was like, just having the quotes searchable, just like command F searchable, like was valuable.

And then, you know, some tools came along to let you do slightly more advanced versions. But basically by the time I got to Google, I had a collection of

versions. But basically by the time I got to Google, I had a collection of 8 ,000 quotes from books that I'd read over the last 20 years. And that

quote collection is like, that's an incredible snapshot of my intellectual influences. Like the ideas from other people that shape who I am today as

intellectual influences. Like the ideas from other people that shape who I am today as a writer and a thinker and stuff like that. And so in the very early days, I kind of had this thought of like, could we build a tool where the AI would have effectively read all 8 ,000 of those quotes and I could

query it and say, you know, hey, I'm writing something new about the human memory system. Have I read anything that's relevant to that? And it would give me a

system. Have I read anything that's relevant to that? And it would give me a distillation of my reading because I've forgotten, you know, 95 % of what I've read, right? Like all of us, right? And so it was really this idea that could

right? Like all of us, right? And so it was really this idea that could somehow I get a tool that was an extension of my memory and could help me kind of just make new connections to things that I've long since forgotten, but that at some point I thought were important. And add to that all the things that I've written, right? Like I had this idea of like, what if I could

give the AI every single thing that I've published, which is, you know, millions of words. And because I also forget things that I've written, you know, human memory is

words. And because I also forget things that I've written, you know, human memory is very fallible. And so that was kind of like in the back of my mind,

very fallible. And so that was kind of like in the back of my mind, like day one at Google was like, could we build a tool that would do that? And it turned out that while most people don't,

that? And it turned out that while most people don't, like, I remember this great conversation I had with Clay, the guy who hired me.

Both Clay and Josh and I, all three of us are quote collectors, as it turns out. And we, you know, you could kind of, it got a lot easier

turns out. And we, you know, you could kind of, it got a lot easier to collect quotes because you could just read in an e -reader and highlight the passage. And then, you know, you could kind of say that. And so Clay actually

passage. And then, you know, you could kind of say that. And so Clay actually had more quotes than I did. He had like 13 ,000 quotes. And I remember we had an early meeting and I was talking about this vision and Clay said, yes, but Steven, the thing you have to remember is that, you know, we are not normal. Like most people are not walking around with 8 ,000 quotes, you know,

not normal. Like most people are not walking around with 8 ,000 quotes, you know, so we just like, we can't design the product like too much for that use case. And I think I said at the time, I was like, yes, it, that's,

case. And I think I said at the time, I was like, yes, it, that's, I know that's technically true, but I think there are a lot of people out there that are what, if you, if those 8 ,000 quotes, I think there's something like 2 million words of information. There are a lot of people out there. Anybody who works is a knowledge worker of some kind or another has

there. Anybody who works is a knowledge worker of some kind or another has a kind of a body of work that is probably about as high a word count. It's probably millions of words and stuff that their work is based on. Like if you are a, if you're a documentary filmmaker, like all the

based on. Like if you are a, if you're a documentary filmmaker, like all the transcripts of all the interviews you've done, if you're a lawyer, all the briefings that you've written and all the legal precedent that you need to do your job, that's, that's a million words, two million words, whatever. If, you know, if you're a student, just thinking about like everything you have to read over the course of a semester

and all the notes that you've taken, all the stuff that might be that size as well. So there's actually a huge audience of people who are carrying around all

as well. So there's actually a huge audience of people who are carrying around all this information that it's hard to organize. It's hard to search. It's hard to get access to this kind of personal knowledge base. And yeah, you don't have to be a quote collector for that to be relevant. And so it turned out that building

a tool that in the very early days was really centered on the needs of one idiosyncratic user, me, ended up being kind of the fastest way to get to a tool that was actually useful for millions of people and tens of millions of people so far. How did you design the team to build such

a tool? Because this is something I always find interesting to look at a tool

a tool? Because this is something I always find interesting to look at a tool and think like, what did they think by designing this tool in order to understand the nature of the technology? It's a great question. So I came into this kind of very naive. Like I had not, I had done a couple of startups in the past. So I had some software history and I think that was helpful. But

the past. So I had some software history and I think that was helpful. But

I was definitely not, I'd never worked inside a large company before. And

so I have learned a lot about how Google works and how Google kind of organizes its products and things like that, that I just had to learn on the fly. But labs deliberately kind of like keeps its teams very,

fly. But labs deliberately kind of like keeps its teams very, small and very flexible. We kind of not by design, but just by lucky accident, we had, we had a lot of people who were actually like humanities majors on the team for a software product. Like Adam Vignel, who was one of the

early engineers is also a science fiction author and very literary. And, you

know, he's a brilliant coder, but he also is very, you know, he's a big fan of like postmodern fiction. And, and so I think there's a, there's a kind of literary, literary quality and kind of scholarly quality to the way that notebook was designed that you can see reflective of the team that, built it in

the early days. But another big thing is that we had a structure at labs that was incredibly helpful in that there were lots of other projects happening at labs that were literally, you know, 10 feet from us or like 50 feet from us. Like we're, so we're like hanging out with these people who are

working on other really brilliant ideas. And, and at several critical moments, we've basically like absorbed some of those ideas from other teams. And the best example of this is the story of audio overviews, which is really how we kind of first broke. The next question. Yeah, yeah, yeah. So that story is, it's just, it's, but it's interesting one in the terms of the history of

notebook LM, but also I think it's an instructive one in, kind of how innovation works in a way. So, um, it's a very funny story too.

Uh, and it's not your voice, is it? It's not my, it's not my voice.

So, so interestingly in the early days of notebook, like one of the early things that I did is I created a style guide for what the AI should sound like. Like you, it's almost like I was starting a magazine and I wanted like,

like. Like you, it's almost like I was starting a magazine and I wanted like, you had to have a style grad for what the magazine should sound like. Well,

we got to make some editorial decisions about like, well, we want this model to sound like, and I had this thing that notebook actually still kind of largely adheres to in, in text chat, um, which is the model should not pretend to be your friend. And it shouldn't even, I kind of discouraged it from having a subjective first person voice even. So it shouldn't say, I'd

be happy to help you, Steven. Like it should just give you the answer. Yeah.

Right. Not say great. And it's, I love that Steven. I'm so happy to be your servant. You know, like I just didn't want any of that stuff. And so

your servant. You know, like I just didn't want any of that stuff. And so

we had a very like austere, I mean, I think it was alienating for some people who wanted to have a friend, but whatever, that was our house style. So,

so we had kind of stuck to that. And in the early days, sometime, but some point in like 20, early 2024, there was another project at labs, um, that was called gem FM. And the idea was you could give the AI some source material and it would generate this amazingly realistic AI podcast between

two people. And it was relying on a breakthrough, an underlying technological breakthrough that Google

two people. And it was relying on a breakthrough, an underlying technological breakthrough that Google DeepMind had created, which is a basically a conversational audio model, which crucially is trained on two people in conversation. Like it's not two different voices that are trained and you put together in a script. It's actually like two people who sat in a studio talking to each other for X hours being recorded and modeled. And so

it had this unbelievably lifelike sense of two English speakers in conversation with each other. It's one of the reasons why it took so long to bring it to other languages, because you have to train it on every language on earth interrupts itself in conversation in a slightly different way. Right. German people

banter differently from English people and you can't just like translate it from one to the other. And so we had to have conversational models in all the languages to

the other. And so we had to have conversational models in all the languages to make it realistic. But anyway, I'm getting ahead of myself. So, um, so we had this underlying technology from GDM and then this brilliant team had built this tool to turn it into podcasts. And when I first heard it, they, they, they had

very extreme personas for the, for the hosts. So it was supposed to be the, sample I first heard was, uh, uh, an education focused podcast about physics targeting kind of high schoolers, like young, maybe like teenagers. And so

each host had a fake physics name. And so the tone of the podcast was, hi there learners. I'm captain kinetic, and I'm here to teach you about physics, whatever.

And so I heard it and I was like, oh, that's amazing. That's really cool.

How fun. That's so lifelike. That's incredible. Didn't think about it for notebook at all.

Like never occurred to me there would be something to put into notebook, but I was impressed by it, but, and, and I could see that there was a lot of creativity and, and how they were doing it, but didn't think about a notebook at all. And then we were about to go to IO, our big annual conference.

at all. And then we were about to go to IO, our big annual conference.

Um, and there had been this question about, um, where we're going to show something from notebook in the notebook had been kind of was still an experiment, but it was public at that point. And they wanted to show this demo of audio overviews gem .fm as it was called then, but it wasn't attached to a product in

gem .fm as it was called then, but it wasn't attached to a product in any way. And with a week to go, somehow, Josh

any way. And with a week to go, somehow, Josh Woodward, the head of labs and Sundar, I believe, like had this brainstorm where they're like, what if we put this gem .fm feature into notebook .lm? And

then, then at least it's not just free floating experiment. It actually belongs to a product and it seems more real. And so there was an emergency call of the team on a Sunday, like it literally, I think it was eight days before IO.

And they said, they said, Hey, we're going to, we're going to like, we think we want to put audio overviews into notebook. Can we make it a real demo that shows it working? And my first thought was captain kinetic. Like I was like, that's not like I, but the whole point of notebook is that you don't have personality in the AI. This is the exact opposite of what I was thinking. Like

that doesn't make any sense. And, and then I took a step back and I thought, wait, notebook is a tool for understanding things. Like that's our mission. We want

to help you understand the material that you need to understand to get whatever job you're doing done. And there's a reason why people like listening to podcasts. And there's

a reason why people learn from listening to podcasts because people have been learning through listening to people have conversations for, you know, hundreds of thousands of years, whereas people have been like reading articles for 400 years. So listening to conversations and learning through conversations is a very deep part of what it means to be

human. And so if we have an ability to take people's source material and turn

human. And so if we have an ability to take people's source material and turn it into an interesting, engaging, stimulating conversation that people can listen to at the gym or while they're driving to work, like that is so notebook's mission. Like that totally fits notebook's mission. We just need to dial the personalities down a little bit. Like,

you know, they shouldn't have fake names. And in fact, they shouldn't have names at all. But so I was like, as long as I can convince everybody of that,

all. But so I was like, as long as I can convince everybody of that, then it's the most brilliant idea ever. And so then basically I was like, I'm sold. And I just want to get out of the way of like, let these

sold. And I just want to get out of the way of like, let these wizards like build this tool and I'll just let them go off. And, and you know, and that was the when it launched, like four months later, it was the most viral thing I've ever been involved with in my life. And it just like took off in this amazing way. So, so it was a great example of my

instincts being absolutely wrong when I first heard the idea. And yet, you know, hopefully, thankfully, we ignored my instincts. Yeah, at the same time, I mean, that's, that's the interesting part of that story. And I didn't know that story. It's a very interesting backstory to understand the nature of the tool, because you mentioned it now, how important it is to not have a personality, but still using that and

turning content into a new format. And I always use notebook as that example, you showed one feature earlier, with the new video options.

And for many people like in our company, many people use it heavily in order to take the source material, and then work with and that's the mode how we call it, you work with the AI to generate something new. And that's, we have three work modes that we call catch up work with grow beyond. And to me, it's not about work with and grow beyond, especially video mind maps, and also the

cards, these are features. And tell me about the video feature you were like, really, like, like a little boy who stumbled upon something new. I'm still saying new things.

So yeah, we have, I'll say two things. I want to say something about mind maps too, because there's an interesting thing about mind maps. So with video overviews, we rolled that out this summer, and it kind of converts your sources into a, almost like a mini lecture, almost like a mini TED talk with slides. And it

will take images from your sources and things like that. But a lot of times people don't have images in their sources. And so the slides were sometimes visually just a little less interesting. And as you may have heard, we have an incredible new image generating model. And so basically, and this is really the

same team that did audio overviews has been doing video overviews. They're just wonderful, creative people. It's amazing. And so they basically said, like, let's, what if you could have

people. It's amazing. And so they basically said, like, let's, what if you could have a state of the art like illustrator for your slide presentation that would do like handcrafted illustrations for each slide. And we're like, we can do this now. And so

yeah, there's an amazing one, which we can probably show some images for folks watching this on video, that is a setting that kind of does cut paper illustrations. And so I've been researching a book idea on the gold rush. And I

illustrations. And so I've been researching a book idea on the gold rush. And I

have this video overview that I generated with it. And it just, like, I just flipped through the slides. And it's just this magical world of like, every single one is just so beautifully crafted. It's just, it's incredible. And it's a great example, again, of like, as you say, like, kind of working with like, I'm curating the ideas,

I'm curating, I'm directing the, in a sense, what the slideshow should be, what the lecture should be, it's all shaped by my vision, I'm trying to, I decide what the style of the illustration should be. So the things that I'm good at, like, writing about history, organizing ideas about history, researching history, turning that into an interesting, compelling narrative that has a message, I'm still doing that. The thing that I can't do

is create beautiful crafted, like, cut paper illustrations for like, every slide. And so great, I now have an AI collaborator that will do that part of my job for me. And that's just so exciting, creatively and intellectually. What really struck me

me. And that's just so exciting, creatively and intellectually. What really struck me when I was, I do a lot of speeches and keynote, and at the same time working on projects, and we have a notebook in the company, because you guys released it very fast as a workspace tool, which is very important. So then you

can use it with the company data with my data. And what struck me was, in areas where I wasn't too excited about, to get an overview, I used it, put the material in, and generated it in a different content format, and made it more entertaining for myself, in order to become more curious on the topics. How

big is that field of boring office work, turning it into a magical research experience?

Yeah, well, Simon, who I mentioned earlier, was the one who said, Notebook finally works.

When he joined the team, and kind of became the head of product for us, he came up with a great slogan for Notebook, which is, make anything interesting. And it's kind of exactly what you say, like, the hosts of the podcast

interesting. And it's kind of exactly what you say, like, the hosts of the podcast are specifically instructed to like, whatever you give them, their job is to pull out the most interesting bits and try to make it interesting. And, you know, it's one of the queries that I often suggest people do, particularly in student or researcher

mode is, which works beautifully just in text. In fact, the models have been very good at doing this for two years now, I would say, which is, upload some new source material and say, what are the most surprising things in this document? Yeah.

And the models have an image, Gemini in particular, I think it's a really amazingly sophisticated sense of surprise and interestingness. And on some level, I was initially surprised by that, as it were. But the more I thought about it, it kind of makes sense because the models, the underlying fundamental

kind of math of the models is all about prediction. And so that's how they learn, is by predicting the next token or the next string of tokens. And so

surprise is, by definition, when your prediction gets, you know, foiled, or you think it's going to go this way, but it turns out to go this way. And so much of the human brain is designed to kind of learn through failed predictions. Like you pay attention, this is how the dopamine system works, right? When you're predicting the world is going this way, and actually the

world goes another way, that triggers a flush of dopamine that causes your brain to like remember that deviation from your predictions. That is like a huge essence of learning.

And so because the models are prediction engines, they're actually quite good at then sensing where like things deviated in a surprising way and sharing that with you now. And

so that way of getting into material, I think, is like as a scholar, as a researcher, is really, it was just never, it was never possible to, you know, command F search for surprising. Like, you can search for that word, but that's not what you want. And you want the concept of surprisingness. And now you can do it. What can you share about, and I'm very curious on that, and

you have to tell me if you can share, because again, to me, this is at the core. How important is the model design or fine tuning the model for a certain case? And I mean, the Google models are very good at retrieval, surprise, surprise. Yeah. But is there a specialized model? Is there a team developing the

surprise. Yeah. But is there a specialized model? Is there a team developing the model underneath the product? How does that work together? We mostly just benefit from the underlying Gemini models as they come out. In the early days, you know, we were really, I think, the first public product that did any kind

of source grounding. I think we were the first, you know, chat with your documents product to be announced or released, you know, because we announced it in 2023, like mid -2023. Which is quite difficult to do that with a technology that is made for generating stuff. Yeah. And it was really, the biggest limitation was the

context window was so small. So, fundamentally, what is going on behind the scenes is Notebook is taking your sources, putting it into the model's context window, which is effectively like its short -term memory, and saying, hey, based on this information, answer this question, or respond to this query from the user based on this

information that's in the context window. So, it has its general knowledge and its training data that happened, you know, months or years before. And then you have the short -term, like, focus on this and answer this question. And, you know, when we first, when I arrived at Google, the context window was basically like 2 ,000 words long.

So, you could put, you know, there was no point in using AI because you could just read those 2 ,000 words. Like, that was like not really very helpful.

You know, now we have contexts that are more than a million words, so you can get a lot of information in the context window of the model. So,

in the early days, we helped a lot with the development of Gemini in terms of giving them examples of source grounding. And we would kind of, like, send them our evals that we would do, and we were kind of helping

them a little bit. And so, I think that there was some nice, like, two -way feedback there that helped Gemini get to be, I think, really the best model for source grounding. It's always been very good at, like, sticking to the facts and the sources you give it. And then we just, ultimately, then they just went off to the races, and they're amazing at what they do. And we rarely can give

them advice on how to build Gemini. What happens is, you know, they'll release a new model, and then we will do a lot of testing because, you know, these models are unpredictable, right? They are, it's truly, like, emergent technology, and you kind of come out with a new model, and you think it's good at a lot of things, but you never really know. And so, every time we shift to

a new kind of version of the underlying models, we have to figure out what has changed in terms of the house style. So, like, one of the things we're always battling is some of these models really like to put things in bullet points. Oh, yeah. And so, and so we have - You solved

it? We have, I mean, we like bullet points. We think the bullet points can

it? We have, I mean, we like bullet points. We think the bullet points can be good. And so, our, the kind of default instructions that you see,

be good. And so, our, the kind of default instructions that you see, that you don't see, but that happen every time you interact with Notebook LM. Like

the system instructions behind. The system instructions, like, we have our own custom system instructions that, in the old days, I used to write all of those. Now, we have more experienced people, but it was kind of nice to have a writer doing those in the early days. That was one of my favorite parts of the job, is, like, figuring out how to talk to the model. But we give the model instructions

to say, if it's complicated and if it seems appropriate, use bullet points to explain the material to the user. And generally, that would generate good results. But every now and then, we would switch to a model and it would just, whatever you did it, you'd be like, write a poem. And it would be like, sure, bullet point number one. And like, no, I said poem. There should be no poems, bullet points

number one. And like, no, I said poem. There should be no poems, bullet points in a poem. And so, there's things like that, or like the model will get very terse or it will write, you know, much longer. And so, you're always kind of, you're kind of trying to, like,

on the fly adjust the system instructions to get the same output that we think of as the notebook house style as the model changes a little bit. And the

other, but the other key interaction there is that they, you know, they come up with some new wizardry like Nano Banana or like VO.

And we got, you know, one of the great things about this job is like, we get an advanced peek at what they're working on. And then, we're always kind of like, ooh, how could we, how would that new feature, that new capability work inside of the notebook product experience?

And, you know, the integration of those images into video overviews is a great version of that. Like, we're like, oh, wait, hold on. I think it could actually illustrate

of that. Like, we're like, oh, wait, hold on. I think it could actually illustrate every single slide in a really creative way now with these new image tools. Like,

why don't we add that to video overviews? And here we are. How is it with context window length of the models? I mean, Gemini models have the biggest context window. There's also research that slightly indicates that the intelligence of the model

window. There's also research that slightly indicates that the intelligence of the model is weaker when you have more context. So, how do you balance it? I mean,

you have a lot of sources. Yeah. And I guess for many people, it's hard to understand that you cannot just drop the whole company into the sources and go with it. I mean, it's not useful to do that, I guess. But at the

with it. I mean, it's not useful to do that, I guess. But at the same time, that's better. Do you have a team? Do you manage that? Is it

like with the instructions that you work on? How do you do that? Yeah, we

now have a whole, it's amazing, we have a whole team of like kind of quality people who are constantly testing the new models, the new kind of options for context. And we, interestingly, like we haven't, we've chosen to not expose what is actually happening in terms of the context, how much you're using.

So you can have, as you said, you can have a notebook with 30 million words in it, and we will use tools like RAG to pick the most relevant passages and present them in a way that hopefully expresses the overall meaning of the documents, and so that it fits in the context that we're using at any given

time. So, we have an amazing internal team that's constantly

time. So, we have an amazing internal team that's constantly trying to figure out like, what's the best model to use? What's the best context size to use? How can we best kind of pack that context if the user goes over? But we don't ever show, there's no kind of slider

goes over? But we don't ever show, there's no kind of slider that says, you are exceeding the context with this additional source, or you're 10x the context, you might want to make it smaller. And we've had a lot of debate about whether we should show that. And honestly, I think I've, I would say I've historically been on the side of wanting to expose more of that to the user.

And I think maybe we're actually coming around to that. We'll see what happens in the next couple of months. We might expose a little bit more of that, because we feel like people can now understand that. And it's relevant to the quality.

But the one key thing that is a really important feature, talk about turning from a tool tourist into a, what was the pro? In German, I call it aus tooltouristen AI -Anwendungsweltmeistermann, so Champions League. yeah, yeah, Champions League. Okay, so one Champions League

tip. You'll see in the source panel, where you have all your sources loaded, there

tip. You'll see in the source panel, where you have all your sources loaded, there are check boxes next to every source. And when you uncheck a source there, it's as if the model can no longer see that source. Mm -hmm. And

so, or if you, in reverse, if you check the source, select the source, the model now will see that source when you ask a question, it will answer based on that source. And so, one thing I do a lot, and we see users doing this a lot, it's kind of an obscure feature, but actually like our power users use this all the time, is when I'm trying to, if I've got a

notebook with a lot of sources, if I'm asking a question where I really know the question is ultimately about like these two sources. I deselect everything, and I just select those two, and I just kind of ask my question based on those two sources. So, it, you'll often get the right answer if you don't do that, but

sources. So, it, you'll often get the right answer if you don't do that, but if you really want to make sure that the model is just focused, it's literally, you should think about it as focus. Like, imagine you're sitting there with your research assistant, and you're like, I really need to understand the information in these two documents.

Would you hand your research assistant like 50 documents and say, read all of these, but I really want you to just focus on these two? No, you just hand them the two documents you want to focus on. So, so I, you know, I think dynamically using that focus feature is, is always a way to feel confident that you're getting the absolute best results. You shared one other thing in

the presentation that I liked a lot, and many people overlook. You can turn a note, you have the three pillars, you have the sources, you have the chat, and you have the creation. And you can turn, maybe explain that a little bit, also what you created into a source, and why it's important to play with these two sides. To me, this is really pro user stuff. Yeah, it's, I use this all

sides. To me, this is really pro user stuff. Yeah, it's, I use this all the time. It is an undernourished part of the

the time. It is an undernourished part of the application right now. I want to invest more in it in the next year, particularly for students who take a lot of notes. So you can write your own notes inside of Notebook LM. It is a notebook. And so you can, there's a button down in the bottom right hand corner that says add note. If you do that,

you just get a little mini text editor and you can write whatever you want.

You can copy stuff in there if you want, whatever it is. Those notes are stored generally in the studio panel. And when you create a note, the model does not know anything about the note that you've created, that you've written. So you have to click on the little three dots to the right of the note. And there

you will see an option to convert this note into a source or convert all of your notes into a source. And I use this feature all the time. So

I'm researching a new book. I've got all my notes in here. For like ideas, for characters, for structure, quotes from chapters that I read, you know, things that I put in there. And every now and then I just do convert all notes to source. And that brings, creates basically a copy of those notes, brings it over to

source. And that brings, creates basically a copy of those notes, brings it over to the source panel. And at that point, the AI knows what I've been thinking, as well as like what I've been reading effectively, right? It can follow my own thinking.

That, I can say this very bluntly, is an incredibly stupid way for the software to work. And it's, it's just one of these things is leftover from the random

to work. And it's, it's just one of these things is leftover from the random choices we made in the early days of the architecture. And we just haven't gotten around to fixing it yet. Someday, your sources will just be automatically grounded if you want them to be grounded. And probably they'll live over in the left hand side with your sources, your notes will be automatically grounded. But until that time, you have

to use this little hack. But once you do that, you know, I was showing you earlier, I have this research notebook for a book about the gold rush. It

has all my notes in it. And I can say like, okay, create a chronology of the main events of the book that I'm thinking about writing. And there are many, many events in the sources that are not related to the book that I'm thinking of writing. But because notebook can see all those sources, and it can see

my notes, and knows what I found interesting, it can generate this chronology of all the events, which is incredibly useful to me as a writer. And that would take a week to do if I tried to do it on my own. And notebook

will now do it in 30 seconds. But that's because I've been one, taking my notes internally, and two, converting them to a source. Awesome. This is really turning into Champions League for sure. Last sentence from you as the co -founder, what is the future of Notebook LM? Yeah, I mean, take us somewhere where you say like, We

didn't even get this shows you how much versatility there is in this product. We

didn't even get to the thing that I'm really passionate about now, which is notebook is a publishing platform for sharing knowledge, right? Like what if, what if for my next book, I publish it in addition to a hardcover and an audiobook? What if

I publish it as a notebook that people could buy and have conversations with that and turn an interesting chapter into an audio overview or whatever? With your book? With

my book. Like I think as a, as a way of publishing knowledge, there's something extremely powerful that we've just started to scratch the surface of with notebooks.

So I think I'm going to be here at Google working on this thing for a long time. I have so much to do. I'm very happy if we have a second episode. Yes, please. Let's do it. Once you're ready with that deep dive.

Steven, thank you so much for behind the scenes. Thanks for having me. Yeah, thank

you very much. Such a, such a treat. Thanks for having me. Bye.