⚡ Inside Google Labs: Building The Gemini Coding Agent — Jed Borovik, Jules + AIE CODE Preview

[Music] Okay, Jed Borovic, welcome to Lanpace.

>> Yeah, thanks for having me.

>> Uh, so we're sitting here at F. Inc's

uh, beautiful podcast studios and we're actually meeting at GitHub Universe. Um,

how's it been so far?

>> It's been great. Um, I mean, yeah, the keynote today was was awesome. It was

fun to see Jules up there a little bit.

You know, we have a lot of folks from our team here. Um, Jules is yeah, partnering with uh, GitHub for the new Asian HQ stuff, which we're excited about. And also this is a this is an

about. And also this is a this is an incredible podcast space. So yeah, I'm excited to do this here.

>> I'm glad for them to to loan us this space. Uh you are also MC for AI

space. Uh you are also MC for AI engineer code. Uh that's exciting in New

engineer code. Uh that's exciting in New York where you you went to college but you don't live there anymore.

>> Yeah. No, I spent a bunch of time in New York. You know, it's funny um being part

York. You know, it's funny um being part of the New York tech scene. I actually

think it's great having big major conferences there. Like I think uh you

conferences there. Like I think uh you know, so there's a lot obviously that happens on the west coast. Um but being someone in tech on the east coast, it's yeah, it's just awesome to have have stuff there. So,

stuff there. So, >> you mentioned you fly over to SF a lot and like like what's it what's the scene like in the east coast like uh like obviously we are pretty new where like

this is our first year coming to New York. Um what else happens in the New

York. Um what else happens in the New York like what are the highlights for you and the New York tech scene?

>> Yeah, I mean there's so much there's you know obviously a ton of great companies.

Um I think the thing that's interesting about New York is it's such a big city with so much going on, right? And so

there's like you know tech is a huge part of it but there's also you know so many major industries there whether it's media sash and finance like it's a um in some ways like I think that helps push

the tech um and do do all kinds of stuff um but yeah no the east coast is a great city great you know the the all the schools are you know all across the east coast um ton of great schools and great students doing all kinds of stuff um so

yeah you know I went to school there hackathon scene there was amazing um really fell in love with tech and and programming there so >> is there a big NYU new hackathon like like uh in Stafford with like Cal Hacks and stuff.

>> Yeah. So there's um >> by tree hacks.

>> Yeah. There's one that was put on by know this is a while ago but we put on by uh NYU and Columbia we do together hack and y uh house. So there's there's uh there's a bunch of events kind of that we did together would bring you

know people across New York City, students across New York City and those were super fun. Yes. So it' be the Columbia one one time then NYU the next and would cycle back and forth. So yeah

a lot of cool stuff was made there.

>> Nice. Uh so you've been at Google for a while, 9 years. Uh you worked on a bunch of things, including with Malta, which which is also another guest that I'm interviewing today. Uh how did you get

interviewing today. Uh how did you get into Jules? Like what's the what's the

into Jules? Like what's the what's the AI journey?

>> Yeah, so you know, this is going to sound really cheesy, but I've told this story a couple times to folks when they're like, "Oh, how did you end up, you know, doing this?" Um but it is actually very true. So I worked on search for a long time. Uh and

specifically kind of like news and freshness. Um and then uh you know when

freshness. Um and then uh you know when stable diffusion came out that to me was the first like geni moment. I know

people talk about like chat GBT as like the first thing but for me stable diffusion um you know it was a couple months before chat GBT came out. It was

a huge thing and I was I was following it a ton online and there were two groups of creators having reactions to it. You know there was one group that

it. You know there was one group that was um you know this is stealing my art.

This is stealing everything that's near and dear to me. I hate this. This is

ruining my life. And there was another group of artists and creators who were like, "Oh, this is a tool to create better art." And so I was watching this

better art." And so I was watching this video brush.

>> Yeah. Exactly. And right around then, um, I was having conversations with a couple people who would say things like, you know, if I had a kid in college, I wouldn't recommend they study computer science. I was like, what? What? And

science. I was like, what? What? And

this was, you know, long before like Jensen Hong and people like he had been saying this kind of stuff. I was like, whoa, what's why? And it was like, oh, AI, like software engineering is going to change. It's going to be so, you

to change. It's going to be so, you know, who knows if they're going to be jobs. And I was like, I loved being a

jobs. And I was like, I loved being a software engineer. I love programming

software engineer. I love programming like and I was like wait this is my stable diffusion moment. This is either it's going to take my my art my craft this is a tool to create better art and I was like I definitely know which path

I'm taking. So I got you know very into

I'm taking. So I got you know very into building coding. was still working on

building coding. was still working on search. Um, but I spent a bunch of time,

search. Um, but I spent a bunch of time, you know, making stuff for my own time and playing with things and um and, you know, ultimately tried to find a role that would, you know, the most exciting role I could find to to do this stuff

and and that was to join Google Labs and Jules where we were, you know, right around then we're starting to build these kind of coding agents at Google.

Um, and yeah, the timing worked out well and uh, I joined and yeah, it's been it's been awesome. Can you uh since we're talking about Google labs, I am actually unclear about where Google labs

starts and the rest of and then then deep mind and the rest of Google like what what is the orchart layer?

>> Yeah. Yeah. Yeah. Yeah. That's a great question. Um so labs's mission is to

question. Um so labs's mission is to build kind of new like innovative products that the rest of Google isn't well positioned for.

>> Yeah. Which we've had like uh riser from Nom.

>> Exactly. Exactly. So is maybe the most widely most and then nano banana. I

don't know if it's >> Yeah. So, um, some of the, so the thing

>> Yeah. So, um, some of the, so the thing about that's really exciting about labs is we work incredibly closely with Deep Mind. Yeah.

Mind. Yeah.

>> Right. So, all the stuff in terms of the, you know, like we're building a product, but we work so closely for the model and you know, one of the nice things about being at Google is you have this opportunity to really build an endto-end AI product, right? From like

pixels on the page through the infrastructure through, you know, the model and the training and all of that loop. So Labs is here to build new

loop. So Labs is here to build new products and we're really like a product org but a true AI product org where we work incredibly closely with with you know deep mind but also you know other

parts of Google you know as it makes sense. Um yeah just on the history of AI

sense. Um yeah just on the history of AI coding I had heard that actually Google had an internal version of copilot or something like that that was never

released. Is that is that true? What can

released. Is that is that true? What can

we say about it?

>> Yeah so you know I think there are there are you know Google's published papers in this space uh for a while. Um and so yeah, we have you know in Google we built a lot of our own tools and you know cider which you know folks maybe

have heard of is our you know our internal IDE and we've had all kinds of you know capabilities and tools there for a while. So yes you know we certainly have had pretty good tools for a while but they were for internal use.

>> Yeah. Yeah. I think I think it was interesting because like I think the uh one of the hype moments when Google started getting into the sort of like LM

game like basically when everything rebranded to become Gemini and like starting to push out Gemini where people are like oh like did you know that Google probably like Google's entire repo is uh probably about the same size

as GitHub and like you know there must be some interesting data in there.

>> Oh yeah. I mean and that's one of the things you know in building a lot of these internal systems is you know the the the data is incredible. Yeah.

Especially when it's you know not only is the model and the training in house but all the data on the usage and whatever. So you know we could build

whatever. So you know we could build really kind of sophisticated sophisticated things there.

>> Yeah. Okay. So let let's introduce people to Jules on your your website says Jules autonomous coding agents. Uh

we've seen lots of these. They're

they're not octopuses. They're not

purple. So you got that you got that going for you. But like what what really is like the core thing you're trying to nail in a very crowded coding agent landscape?

>> Yeah. So what we think about and what we set out to do, you know, back when I joined was like where is where are coding agents going to go and as these models get more and more powerful and sophisticated and um what is that

experience going to be and let's build for that future, right? And so when you think of a really powerful agent that can run for a really long time doing really complicated things, that's when like the products started to shake take shape for us. So for example um

autonomous you know means like it has its own computer right so for jewels it's you know the end exactly so tons of agents that run you know locally or in your workspace with you while you're coding um but if you want something that's going to run for hours or let's

say days you know you might want it to have its own environment where it can can do its own work. Um, so that's, you know, just one of the pieces that that's important for kind of this uh autonomous cutting engine, but it's really like,

you know, think about this future where they're incredibly powerful. You can

spin up tons of them, right? They're

they're autonomous, but also, you know, we we're thinking about what is what does it mean for it to be ambient, right? Like it's it's um kind of when it

right? Like it's it's um kind of when it has its own infrastructure and its own computer and its own ways to interact with it. Um, how does that start to

with it. Um, how does that start to change what it can do? For example, like we have an API. So people are using it for all kinds of things. Triggering it

from you know when something happens and you we saw an example where someone has they're triggering Jules for to do kind of all kinds of updates to their site and then they have a GitHub action that is going to automatically merge Jules

pull requests. So it's just like all

pull requests. So it's just like all kinds of stuff is flowing really kind of changing how people are are able to do stuff. Um

stuff. Um >> and is CLA related just to close that loop.

>> Yeah CLI. We also have a CLI which is you know we want to meet developers where they are right and so part of the you know an API is like you can trigger from anywhere but also you know when you're working locally like you want to

be able to trigger stuff. So um we um Yep. So we have we have the the dual CLI

Yep. So we have we have the the dual CLI we launched a couple weeks ago which lets you interact with it. By the time this podcast comes out we'll be in the integrated with the Gemini CLI. Um

>> that's what I was thinking like you have a number of CLIs. I'm not sure >> exactly. So, Gemini CLI, um, all kinds

>> exactly. So, Gemini CLI, um, all kinds of places where we're going to kind of mix and, uh, and you be able to harness this power, right? Because, you know, developers work in all kinds of spots.

Um, and so making it easy, uh, easy to, to, you know, have this autonomous ambient um, agent that can really do all kinds of work for you.

>> Yeah. What is your journey like when when you started out like did you find any assumptions that were quickly challenged you know when working with geni and coding agents in in general

like I guess you you're maybe not too unfamiliar with it because search uses a lot of like machine learned like blackboxy type things including BERT u which you know was was was a major

update a few years ago. Uh yeah so I mean just just fill us in like what is your AI engineering journey?

>> Yeah totally. So you I think one of the things that keeps coming up is like the the model makes such a difference. I

mean maybe it sounds obvious but it's like the the quality of the model really changes like what you're able to do and how how you engineer around it. So for

example when we started this was you know with relatively early models of Gemini we had the agent scaffolding around it was incredibly complex. So I

think one of the things we've seen is scaffolds get simpler and simpler over time as the models get better and in some ways this the scaffolding is almost a crutch for for uh things the the the

model struggles with. For example like you know really complicated sub aent systems you know we we've played with that we we've experimented with that.

>> Can you give an example of like a kind of sub sub aent that you had to abandon?

Yeah, it was just basically like you have um you know you give you give Jules a coding task to do and it's going to have different agents for um whether

it's um you know making a code edit or handling a sub problem or you know doing any kind of action with an integration or you know having like full sub agents for for different parts of like a reviewer agent or even like people

sometimes people do these like different personas where you're like you know one of the things that cracked me up is like you know you're the product manager agent and then you have the code reviewer agent know the age this video

we we didn't go you know that far um but I think a lot of these things aren't as in favor I mean certainly people do you know they like I don't want to say the agent harness isn't sophisticated it

certainly is but um you know as the models get better like less is more um especially as it comes to like being able to improve through whether it's machine learning or um just you know

regular maintenance um I think certainly we found that you know we're finding that less is Um I think that you know that we were talking about a little bit before we started recording like the um

like rag right and and like you know mixing and all that stuff and you know it seems like you know not just for for Jules but kind of across across the

industry um that like agent based search right like it's you know maintaining embeddings is hard but getting the chunking right is hard like in terms of like the black box aspect you mentioned

um a lot of that is is hard to improve upon and and uh >> yeah, I would even say it's maybe not even hard so much as it will never be good.

>> Yeah. Well, what tell me more why do you say that would never be >> because like a chunk that happens to capture the thing you're looking for um you know will will will fail to capture

something else. And so if you only

something else. And so if you only retrieve based on like your embeddings of a chunk like it it's it uses very arbitrary boundaries that are drawn like

hope with like some hope of like the semantics being captured but you could just throw attention at it.

>> Totally. Yeah. And you can scale probably much better using GP like >> totally totally. So I think that's you know that's an example of of you know in these these harnesses how like they're

simplifying you know.

>> Yeah. Well, I I haven't abandoned it completely because one of the things that we were doing, I don't know if you saw the cognition swap uh work was basically using semant semantic search

and chunks and with embeddings as a tool uh but it on the same level as the other tools like web and and file access and and GOP and whatever else uh other

variants you have. Uh so I think like yeah I mean that that makes sense like don't abandon it. Just just don't raify it into like the only way to do things.

>> Exactly. Exactly. And to be clear, like you know, this is an area of research we're doing tons of work on. Um, and you know, I actually expect, you know, uh, in the coming months, we'll we'll be talking about some stuff we're doing

here, too. But it's, um, it's, yeah,

here, too. But it's, um, it's, yeah, it's it's not the I feel like when we started, it was like rag. It was like embedding based rag. Yeah.

>> Was like the thing everyone did. And

it's interesting to see how it's changed.

>> People asked me for like where are the good code embedding models? And, you

know, I pointed them to like a few like Chinese ones. There are some like Nomic

Chinese ones. There are some like Nomic was working on one and then like we found we didn't need them.

>> Yeah. Exactly. Exactly.

>> Very bitter lesson. So, so, so you know I think like that these this are good things like I think like when Jules came out it was kind of a preview. I'm in the like the trusted testers group. So I got to see a little bit. Um and but now it

feels like more of a real product.

>> Yeah.

>> What's that transition like? Uh is there a process within Google Labs to promote things when you feel like there there's some traction?

>> Yeah, absolutely. So I think the Google labs is not you know about just experiments right so like you know notebook gum as we talk like >> it's not very serious incredibly

successful product money >> it's really um and for us I was kind of a little bit of a turning point um so in in May when we we announced uh Jules you

know it was like great reception following IIO um and that was that was a real moment of us to like turn this into you know a very much a real thing I mean it's thing that we were you know we always intended to it wasn't ever

intended you know I didn't you know talk about my journey like I didn't it was always a goal to build like a real product here um and but for us that that was kind of a very key moment very key

milestone for us um and so yeah now is you know it's very much a real thing you know as mentioning talking before like you know Jules and the you know being talked about on the in the GitHub keynote um uh it's yeah it's certainly

here to stay we're we're um we're we're excited to kind of keep building and expanding Awesome. Let's talk about just

expanding Awesome. Let's talk about just like coding just in general. You're uh

you're coming to MCD AIE code summit. Uh

it's going to be your first time at AIE and and the MC. Uh what do you want to know?

>> Yeah. Yeah. Yeah. Well, tell me why why would someone want to Yeah. This is

Yeah. Well, let's turn it around.

>> Oh, boy. This is embarrassing. Um, so,

so I mean, you know, fortunately, we're in our third year, fourth year now, and we have a bunch of, you know, prior art we can just point people to and say, "Look at our YouTube. Do you like that?

You like this?"

>> There's some great talks. You know, I haven't been before, but I've watched the talks. There's a lot of good stuff.

the talks. There's a lot of good stuff.

>> Yeah. Uh, and I'm proud that it features content from all labs. And basically we are like the and this is a pattern I've seen across my career in terms of like every industry needs it's like focal

gathering points to just like trade tips and stuff. Um so I seen that in

and stuff. Um so I seen that in JavaScript. I've seen that in cloud

JavaScript. I've seen that in cloud native. Uh I seen that in data

native. Uh I seen that in data engineering and I was like probably AI engineering will need something like this. And then I also the the the

this. And then I also the the the concurrent thread to this was I went to a bunch of the academic ML conferences new repres and a lot of them like Europe is 40 years old and it hasn't really changed

and is very focused on academics and PhD students whereas I think really you know the the transition in AI going from research to industry is that it you you gradually sh see a shift uh

unfortunately less open source less papers and more products and more uh startups and >> and closed models and and what have you, but people still want to share, people still want to uh hire, they want to

promote uh their work. Um so they need a place to to do that. You can always do that at your company conferences, obviously IO >> and like GitHub has GitHub and Microsoft has Build and and Ignite, but like there

usually is one place where it's like the industry neutral thing where everyone is on the same playing field and meet the best person with >> and like honestly some people like that.

Uh, you know, it's not like you're not going to be treated as like the VIP and like you know, you kind of have to like earn your spot, but like when you earn your spot, I think like people give that

uh that requisite level of attention better uh because you had to.

>> Yeah. Yeah. Course. So, you know, I've watched I've watched the videos online.

I kind of get a sense of specific, but what's happening between that for someone who hasn't been before? Like

what goes on other than the talks like >> Oh, >> yeah. uh a lot of uh well just

>> yeah. uh a lot of uh well just logistical stuff of like invoicing and like vendor selection and venue selection and like did you know we have like five different pieces of software

to like coordinate okay >> speaker logistics and roof logistics and an attendee so I'm going to go I'm going to sit but uh >> yeah what am I going to what am I going to get

>> yeah yeah so actually uh it's really weird because like as I'm the content guy for AIE right I I c the speakers I invite Right. And uh but I actually know

invite Right. And uh but I actually know that the content is like the least important part because all of it's filmed and we're going to edit it and post it for free on YouTube anyway.

>> But the reason you come is because one, you can talk to the speakers, but also you can talk to each other. And so like the the you know, I always say like the hallway track is the most important track.

>> Yeah. What's how do you get the most out of the hallway track? What's your guide be hallway track?

>> I don't have as collective a thoughts as I should. One, I think if you have some

I should. One, I think if you have some prior history of like what you're interested in and work on. So basically

like the best intro to somebody is if they've seen you online before so they can skip the whole like who the hell are you part and just get into like >> hey I saw you wrote that thing like let me talk to you in person about this

since you're both here. Um that's way better than like who are you? What do

you do?

>> And and that's and that's like a very cold interaction. Ideally people come

cold interaction. Ideally people come warm or they can come with some clear idea of like here's here's why I'm here.

Here's here's what I'm looking to get out of this. Uh because if I think if you show up with like no uh real intention or if you're like in and out for uh for your thing and nothing else

then you don't have the space and the mental energy for the unstructured serendipitous connections. And the thing

serendipitous connections. And the thing about at least in at least in our scale our size right now especially for the summits which which is the one that you're going to um everyone had to apply

to get in. Uh so usually uh you know our first um summit we had like something like a 10 to1 applicant to to invite ratio invited spots ratio. This one's

going to be and when it went up to like 10 to 16 to 20 something this one's going to be 23. Um so one out of 23 people who didn't Yeah.

>> So so like uh yeah it's it's it's a lot.

Um I think like um and and really we're trying to filter for people who would be speakers at any other conference but like they they are they are the top of the the field. They are either founders

or honestly enterprise buyers of the the best companies you can find in New York.

uh which you know and then that's another reason for this the our New York conference which is we're bringing kind of the best of San Francisco or tech

>> to the the the finance sector really >> uh there there is a little bit of media but mostly finance >> and like yeah that's that's great like I mean I I think so what I'm trying to say

I guess is you're there to meet the other people so make time to meet them have a calling card like >> who are you like like a quick like what who are you what do you do what what can you help with? What are you looking for

help for?

>> That kind of intro stuff is really good.

Going with friends is really good.

Obviously, like we actually offer we uh for the Walter, we offer bundle discounts. This one I don't think we do.

discounts. This one I don't think we do.

Uh but just reach out if you need something. Uh but yeah, I mean like uh I

something. Uh but yeah, I mean like uh I I think like the idea of getting immersed in the code agent community is really important. Uh we and I think

really important. Uh we and I think maybe the last part I'll bring up is that we themed it for the first time, right? So you used to be these are just

right? So you used to be these are just generalists. here's the state of AI, the

generalists. here's the state of AI, the best speakers we can get at any any point in time. But now we're really trying to push ourselves to theme everything. So we have the best people

everything. So we have the best people in code, the best people in data sets, the best people in RL. I want to do a mech and turp one.

>> That'll be fun.

>> Cool.

>> Uh that one that one I I'm thinking will be in London because um the people I want to target are in London.

>> But yeah, I I think like when you do a summit, it should be focused. Everyone

there should have an agenda of like trying to learn what's the what's the state-of-the-art trying to have off thereord conversations with their peers doing the same thing at the other companies and who knows what could happen like that's the that's the

weirdest thing like I organized a thing and I don't even know half the things that go on just because my job is to provide the nexus of people to just connect >> last time we were in New York there were

13 >> maybe 15 side events organized by people just like dinners meetups, whatever around this around the summit and we encourage it. We we post it and we just

encourage it. We we post it and we just want people to meet up.

>> Yeah. I was going to ask is there a whole like off-men set of events happening? Like how do people know?

happening? Like how do people know?

>> Yeah, they uh they they organize it.

Honestly, if you're if you're not scared of strangers, you should organize your own like a little dinner. Uh we we leave all the evenings open.

>> Okay.

>> So, just like organize a dinner or a meetup. Focus on your thing. Like we

meetup. Focus on your thing. Like we

have people doing only voice.

>> So, if you want to do voice, great. uh

if you want to do like code review agents as as a as a small subset of generalist coding agents do that and uh I think you find it right like or you can do like AI in finance AI in bio

whatever whatever that the this particular sector might be um and I think like that is honestly the highest signal way to get uh a bunch of people who really resonate with your thing to

uh to meet and and have like high bandwidth conversations.

>> Yeah. Yeah. Are you and me going to do the autonomous uh coding agent dinner?

>> Uh well, no. Uh my job is to float.

>> Yeah. Yeah. My job is to handshake, ask ask how everyone's doing uh if I fires.

So I I tend to just leave myself open open until you know the end. But yeah,

it it'll be it'll be a sprint. It was

it's it's always a mad rush because then I have to do my own talk. Uh and

>> uh I don't know yet. I I think um so far so like the last time I did this summit I was talking about how this year had to

like develop as the year of agents >> and like it's really played out a lot obviously now you know the trendy thing is to say it's no well it's not just the year it's the decade of agents but like um this year I think agents really took

off and most people got it right like the consensus was correct you don't have to be too spicy or counter consensus to say like if you worked on an agent you're probably a lot better off you probably made a lot of progress this

Um, and maybe you can tell me how it feels from the Jules point of things. I

didn't see myself at the start. You're

joining an agent company >> and I ended up doing that. And but like I I've gone so agent pill to the point where like people come to me with startup ideas for infra companies.

They're like what if we made a agent framework so that other people couldn't build agents. I'm like why don't you

build agents. I'm like why don't you just build agents yourself bro?

>> Like >> there are a lot of frameworks. Yeah.

frameworks and infra companies >> and all these guys are just like they're good developers with no conviction whatsoever in what they want to build.

They don't know what they what customer they want. They're just like

they want. They're just like >> we want to build developer tools. So

that's where we feel comfort comfortable.

>> But honestly, it's not that hard to actually take a stand and be full stack and verticalize in some particular agent field that you want because guess what?

they like the the business and the economics are are you know aligned that way >> and I'm not saying that you cannot make it as an infra company. There's some

fantastic infra companies that u our sponsors and like that I I admired and you know I would invest in myself. It's

just that comparatively those are a lot harder and like agent companies seem like they're shooting fish in a barrel. They

seem like they're ramping up in AR a lot faster and you seem like their margins are better so why not?

>> Yeah. So I mean I think for us it's certainly been you're the agent like as the models you know as we were talking about like what is let's build jewels for where things are going and as the models get better I think it just becomes clearer and clear that agents

are super powerful you know like we have um uh you were talking about like before high context and management all that stuff's important like we have people we had this is a funny story we we store

some data for a session but it only lasts we we only store it for 30 days and so after 30 days uh your your session becomes locked And when the first user start first started hitting that, they were upset. We're like,

there's no way anyone's going to be using a single session for 30 days. Like

people doing a single track of work for 30 days. Uh just like how powerful that

30 days. Uh just like how powerful that could be. So yeah. How do you compress

could be. So yeah. How do you compress context when you run into it?

>> Yeah. So we have I mean I can't talk too much about it, but we you know we do a lot of the standard things and it's also um you know we're developing a bunch of stuff. It's an active area research for

stuff. It's an active area research for us.

>> Yeah. I I think like uh you know just just to I'm not not asking you for for how exactly Jules does it. There's just

a number of approaches, right? And you

just have to pick one because you can't just use up your 2 million token context window. Is it is it 2 million? It is up

window. Is it is it 2 million? It is up to 2 million. especially for coding agents because like you're reading files like it's so you're you're running commands with huge outputs like you know I think coding agents are a really interesting area both product wise and

the impact they're having but also for research like they really push the limits of you know what what other domains are you running an agent for 30 days what other domains um are you accumulating so much context and so many

turns and um so it's uh yeah it's the coding agents are I think kind of a special spot of like super interesting product impact research >> yeah I I see the M folks drop the auto

compaction for a handoff mechanic which was pioneered by the agents SDK >> which is basically the sub agents pattern where like you spin up a subgent do do a thing you don't need all their context that sub agent is doing

>> and then you you can sort of come back to the main thread >> totally yep yep yep so it's yeah it's a a good pattern also has this challenge is like how do you make sure enough stuff you information is going back and forth but that's probably you know the

summarization is a pattern um you know like kind of externalizing some of that context whether it's like writing it to you know like a note kind of thing is is a common pattern. So yeah there's there's tons of things uh tons of things

to try and do.

>> Yeah. Yeah. I mean and one thing I I do want to get more consensus about is what is the best because I don't think I've read any papers >> about which uh methods compare better.

>> Yeah. Like as models change like the answers change a little bit too.

>> Yeah. Yeah. Cloud you probably know cloud externalizes too much.

>> Yeah.

>> Yeah. Y

>> uh how much does your work actually like I feel like I switched back to to Jules mode.

>> Yeah. Yeah. Keep free flowing here.

>> Yeah. Well, I mean like you know how much does your work inform the model creation, right? Like at the end of the

creation, right? Like at the end of the day like you obviously are a very big consumer of Gemini models, >> but also you are not the only consumer and they have other priorities than you.

>> Yeah, totally. Totally. I mean I think we're lucky in kind of how we're positioned to we we have very close relationships with with Deep Mind. But

we have um and you know coding agents are an important area like let's be honest right like for any kind of company building models like you can see it in all the labs like coding agents are important coding capabilities are

really important.

>> Yeah. My my OG image of the AIE code u I I wrote something obnoxious like code is the first spark of AGI.

>> Yeah.

>> Which is like probably true.

>> Totally. Yeah. Uh it's important from kind of a AGI perspective. It's

important from a dollars perspective. is

important for all of it. So it's um I think we're in a really lucky position where we have uh we're able to have a lot of kind of good collaboration that >> yeah but both ways you know like all kinds of capabilities that are being

developed and and you know it's interesting it's it's >> it's a whole host of things right because you know in terms of like AGI and the capabilities of things it's also like computer use models and browser use models and so it's it's a you know

models that output code but it's also the whole suite of you know things that you'd want an intelligent agent to be able to do. Um so it's uh you know multimodal you know it's all kinds of

stuff um that goes into it. So it's yeah >> what would you want to find out from your peers at other coding agent companies because you're you're going to meet all of them basically.

>> Yeah. Yeah. So I think one thing and you know I don't think of this as a zero something. I think this is like really

something. I think this is like really like uh there's this tide that's going to lift all of our boats and um it's we're inventing a new way to do our art,

right? You know, um and how to create

right? You know, um and how to create good art as a software engineer and so what does that look like and how does that feel? What does that you know what

that feel? What does that you know what is the experience we want to create? I

think as as people working AI sometimes we don't do a good enough job describing this beautiful future we're creating. I

mean I I know like you know like the CEOs and heads of these labs have started like you know writing their their think pieces on this but right >> um you know for software engineers like what is this beautiful future we're

creating and like you know I think that's like one it's it's inspiring it makes it you know maybe less scary for for people who are who are thinking about these tools but also like you know if we can't articulate it and think

about it it's less likely we'll get there right so like what is this you know great place we want to create like writing software is so hard like at so many companies it's such a especially like big companies, it becomes so challenging to manage a codebase and

create um and and what can we do to make you know being a software engineer in absolutely incredible experience. What

are these you know how do you want to interact with your model? how do you how are you doing things locally versus in the cloud and how does that interop and um so I think like as an as an industry we're trying to like you know it's

changed like we're invit some ways inventing and there's this movement to you know change how we do our art um and yeah the more you know the better we can create this experience like we all we

all win to some degree um so uh yeah I think that'd be one thing where it's like yeah >> yeah the local to cloud sync is um the most contentious or important I guess

topic for a lot of people. I wonder if we'll ever get like some kind of interrupt thing. Probably not. But Mac

interrupt thing. Probably not. But Mac

and Dream, >> tell me more about what what's your dream? What's your dream flow here?

dream? What's your dream flow here?

>> I don't know. Uh start with dual CLI end up in Devon. I don't know.

>> Oh, interact between Asian guy.

>> It's probably it's probably meaningless.

So, no, but like I'm not actually serious about it, but like >> traffic.

Well, so uh I think codeex or is it cloud code web?

>> Mhm. can do this teleport.

>> Yep.

>> Where they just basically dump like the entire history and you can you can pick it up in cloud code on on your desktop and probably that's the right move.

>> Yeah.

>> Maybe there's there's some more sort of elegant things but they were first so like why not? Yeah.

>> Uh and like and actually maybe the the maybe the real thing is maybe it's not the conversation maybe you don't need to teleport if the unit of if the artifact that you pass back and forth is the

linear ticket or the GitHub PR.

>> Right. So you don't need the full JSON.

Uh you don't need the full chat history.

You just need to pick up where other people left off because that's how humans do it.

>> Right. Right. Right.

>> Right. I don't I don't transfer my brain state to you. I just tell you what I did.

>> Yeah.

>> And then you know if I didn't if I forgot to say something, you find out eventually. Right. Right. You say like

eventually. Right. Right. You say like the cloud agent like dumps some kind of summary onto the ticket or whatever kind of it needs to pass on to the next >> in slack line and whatever.

>> Yeah. Yeah. That's interesting. I think

we have there there are some patterns emerging though like IDE CLI cloud right like these these are the pieces VS code extension.

>> Yeah. VS code. Yeah. like whether you guys don't have like like there's a >> like the surface area is like standardizing it feels a little bit um

and um yeah how these things interop how you can kind of make this like great experience between all of those. Yeah, I

think it's really interesting.

>> Yeah. Yeah. I think like and then the other point I just want to backtrack a little bit to something else you said which is like what the the thick pieces that the >> these cos and stuff do. Uh I I think

there's a lot of question about the impact that co coding has on the software engineer industry in general the humans.

>> Do we end up do we stop hiring juniors altogether? Do we uh is it actually

altogether? Do we uh is it actually increasing productivity or do you just feel like you're increasing productivity? I don't know if you have

productivity? I don't know if you have any take on that stuff.

>> Yeah, totally. So I mean I this something we spend a lot I spend a lot of time talking and thinking about with folks and you also also spend time talking to people at companies and you know I think sometimes working on these

tools it's interesting to see uh it's not as like diffused this technology isn't as diffused across software engineers as I sometimes expect right there's plenty of places that I think

are are not really using AI a ton a lot of companies a lot of software engineers aren't. Um, that being said, I'm very

aren't. Um, that being said, I'm very kind of excited about what this what the future of software users are. Like,

could you imagine going back to not having these tools? No, that sounds horrible, right? Like that. Um, and so

horrible, right? Like that. Um, and so so that's one aspect of it. I also think um, you know, I don't really >> buy this like, you know, that we're not going to hire more software engineers

story. I think like for for a few

story. I think like for for a few reasons. Um I mean this is an example

reasons. Um I mean this is an example that that often comes up but is it like kind of the elasticity of the demand for software.

>> Okay.

>> Yeah. Jevans paradox.

>> Exactly. And you know like a lot of the cases sometimes come up is you look at like farming right and so you know there was a time in America where like the vast vast majority of Americans were farmers right and then technology

happens and today it's like less than 1%. Yeah. Um and that's one example but

1%. Yeah. Um and that's one example but the flip side of that is you you electricity which like as that gets cheaper and cheaper people just consume more and more and more electricity. Um,

and you know, with food, there's only so much food we're going to eat, right?

There's there's a kind of a there's an inelastic demand for that, whereas just very elastic demand. It seems like software, you know, software keeps getting better and better. Like the

ability like we're creating more and more software from like obvious like punch cards through to where we are today is like remarkably different in terms of how you're able to create software. U so much more software is

software. U so much more software is being made and software just keeps becoming more and more of our GDP, right? like it's it's a um so I'm I'm

right? like it's it's a um so I'm I'm I'm bullish on kind of the the amount of software we'll be able to create, how it'll be created. I think there's also something here about, you know, as a as

an engineer being able to be more productive like encourages more investment in people building software, right? If it's, you know, the job of a

right? If it's, you know, the job of a software engineer can now, you know, they can do 50% more, 100% more, 10x more, um, like justifying investment

dollars into projects like dramatically changes, right? And so, um, uh, yeah,

changes, right? And so, um, uh, yeah, I'm I'm I'm bullish on this idea that that this actually going to be great for soft both for, you know, our ability to kind of do our craft art, but also just

what it means for the number of companies and the amount that's made and the quality of it and what we're able to do with it. So, uh,

>> yeah, that's my rose color glasses take.

>> Rose colored glasses indeed. Um yeah, I have this take on the different kinds of work like we we're splitting up the different kinds of software work and there's a lot of commoditized work that we used to spend a lot of time on that

and now we can basically entirely delegate to agents >> and then that leaves us ideally for more strategic important novel high-risk

whatever uh work deep focused work that uh you know is is is something I I I posted here on the uh semi async value of death where basically you kind of need on the on the extreme end you can

delegate to async agents which jewels uh you know uh cloud code whatever but then over here you kind of need the sort of deep involvement in understanding the codebase and like

>> feel like not vibe coding >> whatever the opposite of it is actually that's my talk which is I've been thinking about this okay >> so I tweeted out like uh this phrase cuz I I think I feel it's in the air that

like the term vibe coding was obviously coined by Andre and he's super influential in February and like people have just come to kind of use it as a

blank check to just yolo on prompts and stuff and and create the worst code imaginable and like leave other people to clean it up. Yeah.

>> Uh so I think like people are kind of at their limits with this like it was probably maxed out in terms of popularity and but we don't have yet what's next >> right?

>> So my talk is really challenging every attendee every speaker to come up with like what is the aspirational good version of V coding that we can actually trust.

>> Yeah. What is it?

>> Well the punch line right now what is it? I I mean uh the current leading

it? I I mean uh the current leading candidate is agentic coding >> which is what DSH Sha who's like I don't know if you know who DSH is. He's he's

pretty good track history when he's naming things.

>> Uh it's just too many syllables. I don't

think it just has the it doesn't have the joy that that vi coding invokes which uh I think people want but then uh people also want care and craft and like reliability and all that stuff that >> but if we don't have the term like

describe it maybe we don't have a catchous phrase for it but what is what is what does it look like even if we don't have the phrase >> that yeah it's a great question um I well we have some speakers who are going

to be pitching spectriven development that you have to really >> be thoughtful and effectively write a PRD I think the I think like that is obviously very correct in terms of like

basically it's just a glorified prompt but a very very very good one >> and uh models are tuned to follow your prompt for good and for worse >> if you prompt sloppily you're going to

get slop >> so a spec sounds good I think uh I don't know how often it'll be followed in practice >> because effectively what that transitions us to is a waterfall

development approach where you spend 3 days writing a writing a 50-page document and and you kick off the agent.

That doesn't seem right. Uh so like you know obviously I I I have some bias here because uh cognition has from the start believed in interactive planning where like you kick off a thing you get some feedback then you're like you're like oh

that's not what I meant. Let me correct myself >> cuz I don't know what I wanted when I when I started. So you you work with the a the machine to discover what you wanted and the machine works with you to

either get you what you wanted or show you the errors of your ways and and then you correct it from there.

>> Yeah. I mean one thing we talk about which very align thinking is like they're kind of like two problems as these things get better. One is like how do you specify what you want and the other one is how do you verify that what

you got is what you were thinking.

>> Yeah. Yeah. Um, and so, um, yeah, whether it's, you know, specifying through a spec or this like, you know, interactive plan or whatever it is, but then, yeah, and then on the flip side with the vibe coding thing is you might specify, but you're never to come back

and verify, right? Like you're it's more hands off the wheel like maybe I'll click around the app a little bit and see how it works, but it's I'm not really engaged with the code. So, how do you Yeah. How are you verifying and

you Yeah. How are you verifying and making sure that it's, >> you know, >> to my knowledge, you guys don't emphasize tests that much, right? It's

not like you volunteer to write my tests.

>> Yeah. Yeah, I mean it dep it depends like we um if there are tests in your codebase um it's right it's right out of the the picture here. Jules will run your test feed.

>> Exactly. Exactly. So um

>> but it's not like it's not like you know after everything must have a matching test to the prompt that was mentioned.

You know that would be the extreme of >> what we mentioned.

>> I don't know if people always want that.

I mean maybe it would be helpful to do that to kind of show that was right. But

let's say I don't write tests in my code base like I want to merge that pull request that is introducing tests just for this one thing. like you know I think in in some ways the the the engineer should be able to control what

what kind of outputs they want if it if it's helps and they want it you know absolutely um >> and then do you think there's other innovations on specifying apart from just chat >> oh totally totally um I mean

>> agent agents MDE >> yeah agents I mean uh spectrum development I think is in this this category I think um one of is like multimodal

>> right like you know if I'm going to show a bug on our website like do I want to come and like type it with words to describe it or am I going to point >> at the picture?

>> Yeah. Um and so you know with Jules you can upload images now. Um but you know kind of more you know we have certain ways we communicate as humans that are easier in certain situations. Yeah.

Let's bring that to to our engagement with with the so of all people I expect you guys to be best at this because Gemini has video understanding. just I

want to submit a video because some things like do cannot be screenshots.

>> Yep.

>> Uh it's more about the behavior of of things uh appearing and disappear. Uh

yeah, I mean I would love that if you you guys did it >> cuz no one no one has it yet.

>> I know. I would love it too. We'll I'll

tag you now.

>> On my side the vision the the version of that that we're exploring is computer use.

>> Yeah. uh computer use was kind of introduced by enthropic and then openai did their toe in with operator and now agent mode in in atlas. I don't know if you guys have done anything super

splashy on computer use but anyway it's coming back I I can feel it. Yeah, you

know, I think uh yeah, definitely. And

you know, it ties into, you know, it ties into coding agents, it ties into just, you know, using AI systems in general.

>> But basically, your VM now needs to render a UI or or a browser and then you need to let the agent click around in it.

>> Absolutely.

>> And you need to have precision >> and speed and cost and like, you know, affordable cost.

>> Yep.

>> It's a lot.

>> Yeah. No, these is I mean, what kind of projects are so fun? There's just so much to build. There's so much, you know, I think also as a software engineer working in the space. Like I

think one of the reasons we, you know, you see so many companies in the space is partly like it's just so fun. Like

there's so many things to build. There's

so many tools um that seem like, you know, fun sci-fi. Like there's it brings up a demo of what I've worked on. It's

clicking around and I can see a video of it or I can even take over and use it like so. Uh

like so. Uh >> yeah, it's Yeah.

>> Awesome. Okay. So, just moving towards wrapping up. If people run into you at

wrapping up. If people run into you at AIE, uh they've, you know, they heard your your your pitch on Jules. Yeah.

>> What else should they also talk to you about? Like, you know, what what do you

about? Like, you know, what what do you what can you help with versus what are you looking for?

>> Anyone should feel free to come up and talk to me, you know, at any point. I

uh, you know, obviously very interested in anyone who's doing stuff with coding agents or someone who's using coding agents in an interesting way. I'm always

curious about, you know, workflows people have with their agents. Whereas,

you know, whether it's, you know, hey, I'm using this tool in this way and I've, you know, configured this crazy thing. Like, I always love hearing how

thing. Like, I always love hearing how people are using it. I also love hearing people who are having bad times with it where it's like actually I don't you know maybe they're not coming to this conference but you know I I have tried all these tools and I don't like them

and I don't use them and here's why. Um,

yeah. So, you know, I'm totally open for any side of of the uh all the way from, you know, full AI pill and coding coding AI lovers to people who hate it. As far

as what I'm looking for, you know, I think, um, you know, really just going to kind of connect and meet people. I

think, you know, we are always hiring.

So, like, you know, I'm I, uh, anyone who's, you know, interested in working on this stuff, um, I'm always happy to talk. But, yeah, really just kind of,

talk. But, yeah, really just kind of, you know, meeting people, spending time geeking out on this stuff.

>> Yeah, there'll be lots of geeking out.

>> Uh, all right. Thanks for your time.

looking forward. Yeah, same.

[Music]