Julie Zhuo and Dan Olsen: How AI is Transforming Product Development at Lean Product Meetup

Well, let's get to the reason that we are all here tonight. Julie Zoo. I'm

super excited to have her here for the first time in Lean Product Meetup. She's

been on my list for a while, so I'm excited to have her. She is a co-founder of an awesome startup called Sundial.

Hopefully, she'll tell us a little bit more about that. Before that, she had a long career as a design leader at Meta.

Um, and most of you probably know her from her amazing book, The Making of a Manager, which is a bestseller. there

aren't that many books that tell you how to be a good manager and this is like one of the top books and I know she'll tell us more but there's a new edition coming out learn more about that very soon she also writes on Substack at the

looking glass and I really like her articles they're very insightful um so you want to check that out and we are going to have a fireside chat about how AI is transforming product development so with that please join me in welcoming Julie Zoo

hey Julie thank you all right is your mic on uh thanks sir all right cool me all right Um, maybe not.

No, I had mine off. And by the way, there's the slido code. Right. So, while

we're talking, is it on?

Let me see.

Yeah.

Yeah, there. Now it is. All right. Cool.

Awesome. Welcome, Julie. How's it going?

It's It's going great.

All right.

How about with you?

It's going great. Anytime.

How's it going, guys?

Awesome.

Anytime we have a meetup, it's great.

It's so cool. All right. So, let's talk about the book. Can you tell people that like the first edition or don't know about it what's coming down the pike very very soon? What's the new edition?

Yeah, so um the new edition is coming out on September 9th and it's uh it's it's just a a light refresh of most of the content. But the thing that I've

the content. But the thing that I've been feeling for the last couple of years um as we've gone through many events like the pandemic and a bit of a contraction in the tech industry is the

things that I didn't get to put in the first edition of the book. So probably

the two major things that I always kick myself about is I didn't talk about managing remotely which became a really big thing. Uh so I have a chapter a new

big thing. Uh so I have a chapter a new chapter about that. Um as well as managing during um a contraction/recession

slash difficult period where you might have to deal with layoffs or deal with um you know just going through a tough time in the company. And I know that we've seen that it's been a lot rockier

through Silicon Valley over these like past five or so years. Um, so I'm excited to at least talk about those and address those for newer managers who are are going through these things for the first time.

Cool. And the new edition, this is a hard cover. The new one's going to be

hard cover. The new one's going to be paperback.

Yes, the new edition is going to be paperback. So there will be a update to

paperback. So there will be a update to the hard cover as well. However, if you want to guarantee that you're getting the new edition, you um it'll be the paperback is probably the best bet. Um

they kind of have to like sell through the hard cover before they get the new versions of it. The Kindle

version will be also updated uh in September as well. So, the Kindle version, the paperback will be the newer version.

Cool. Great. So, thanks so much for that. And actually, do you want to just

that. And actually, do you want to just touch quickly on do you want to share any nuggets on managing remotely? on

managing remotely. I mean, uh, you know, Slack's great. Um, I think the most

Slack's great. Um, I think the most important thing probably is just it everything has to be designed with a little more intentionality. That's

probably the first one. So, a lot of the the challenges of of remote management is that you just don't really get to pick up on the cues of like what's really going on for other people? Like,

are they in alignment with me? Are they

actually feeling good about what we just talked about? Do they even like their

talked about? Do they even like their job? Like all of those it's much easier

job? Like all of those it's much easier to get in person, you know, when you can feel the energy of somebody else and or even just you're like talking to them during lunch and you can just ask them

how they're doing. And it's much harder to do because it kind of feels awkward to do it during Slack. Um or or to, you know, to bring it up if there's like, you know, everyone's like jumping into a

room and now you feel like you have to start the agenda. And so I think a lot of how to be more successful is just like tell yourself that hey we have to do these things intentionally. Like we

have to do check-ins intentionally. We

have to have one-on- ones. And if I'm going to talk with someone one-on-one, like I'm going to explicitly ask you ask them things like how's their energy?

Like are they feeling engaged? Are they

is there anything that we talked about in larger meetings that they disagreed with? And and just try and be much more

with? And and just try and be much more explicit about these things. Uh because

otherwise, you know, you can I mean, this happens to me too still all the time. Uh where I'm sometimes like

time. Uh where I'm sometimes like surprised because I didn't realize that this other group of people was feeling this way because it just isn't as obvious when you're not there.

Right. Well, and it makes sense if you if you're a manager and you have one-on- ones, you have that one-on-one forum where you can ask that.

Yes.

As you were talking, my first thought was cross functional partners like you're a developer or you're a designer or you're a PM.

But like then I'm thinking, do we have time to do a check? like how do we balance the time? Everyone's like back toback meetings. Like how do you are

toback meetings. Like how do you are there efficient ways to do check-ins? I

guess with cross functional co-workers.

I I think there are I mean a couple of the tactics I talk about in the book are like sometimes, you know, you have a team meeting, you might spend the first five minutes just having people and what we do sometimes um in our all team

meetings like we'll spend the first five minutes, we'll have a prompt and everyone just like puts a little sticky on our collaborative whiteboard and sometimes it's just a check-in like how much do you believe in our current mission? What are you most worried

mission? What are you most worried about? you know, like what's the biggest

about? you know, like what's the biggest problem right now that's like got you down? And so then everybody can just

down? And so then everybody can just it's like a more collaborative moment where everyone's like, "Okay, this is our time to talk about some of this stuff." So that's one.

stuff." So that's one.

I think the other is just sometimes like I also do a lot of like direct message Slack check-ins.

I do have a one-on-one and you're like, "Oh, but the one-on- ones in 5 days."

I I just want people to feel like it's comfortable. And so sometimes I'll

comfortable. And so sometimes I'll I'll just like try and just send a bunch of messages and just it's like a little touch point, you know, like a more personal touch point. So then if they want to tell me something, maybe it's a

little easier to do. So

nice. Nice. That's That's pretty cool.

Cool. And I like the uh I'm thinking I'm envisioning the team members filling out the sticky notes on a collaborative whiteboard like mirror or something. And

the other cool thing there is they don't have to necessarily vocalize it.

Yeah.

You can do parallel. That's a problem with speech. Only one person can talk at

with speech. Only one person can talk at a time and it takes a long time to go around. So that's a good way to be more

around. So that's a good way to be more efficient. And also people that may not

efficient. And also people that may not feel comfortable speaking up can write it down at least.

Yeah. I've I've even uh experimented with even like doing polls where it's like here's a question and then there's like a slider and people put a little like a little sticker of their face on whether they agree, you know, between

like one and five.

I when you were talking about checking I was thinking about is there any way to like have some like emotion like you know you picking like like inside out you're picking which emotion you're feeling right now or something. Your

your your slack status gets set to angry or happy or whatever. Yeah. Interesting.

Cool.

And then how about managing in a downturn since a lot of people are dealing with that. any highle tips there?

I I I have like a playbook for like look if you're going to have to go and do the unfortunate thing of manage like one of the hardest things I think in management is to have to let someone go especially sometimes with a layoff where you don't

feel like it's your choice and you're kind of have to support what the company is doing. And so I really go into like a

is doing. And so I really go into like a very concrete playbook for what you do like what to say and and how to do that.

Um I think I think the the thing that's interesting is that um in a downturn I actually think that there's a lot of different um like depending on where you are like are were you the decision maker

or are you somebody who like decision makers passing information down like it's honestly it's like all roles are tough but it's like tough in different ways. Um, I think that if you are, for

ways. Um, I think that if you are, for example, a middle manager, it's actually one of the toughest because, you know, you kind of have to execute a lot of this, but and people are asking like your your people on your team are asking

you, you know, for answers, but you may not actually know what's really going on because a lot of these, you know, decisions aren't really yours. And so, I think that can be an especially tough

place to be in. Um, I do think one high level thing is just like it's okay. Like

be aware of the feeling and and that what you're going through is like pretty hard and kind of shitty. Um, and that everyone's going through that, right?

Like it's, you know, obviously the person who might be your boss who's deciding this is also having a very hard time and the people below you who are, you know, who might get laid off is

having a really hard time. I think a lot of it that's hard for us is to try and just be okay with the fact that people are having a difficult time with it and

just allow like what whatever feelings there are and then just to call call it what it is like you know just to be direct empathetic um try not to

sugarcoat too much I don't think that really helps um and just to do it with as much like grace and empathy as we can.

Cool. All right. All right. Well, thanks

for hitting those topics from the new edition of the book. Um, why don't we switch over to talking about AI, everybody's favorite topic and how it's transforming product development. So, um, when you what have

development. So, um, when you what have you seen so far when you think about it, what are some of the things that you're seeing like that AI is making different?

So, I think one of the most interesting things I've seen and I see this a lot with um some of the, you know, more AI native type of companies. Uh, and I'll

I'll give you one example like I spent some time recently with Perplexity and their head of design and I got to see a little bit of like how are they operating and you know they're obviously

building a you know a very AI native type of product and I think they're also trying to operate in a very AI native type of way and I think one of the things that they said and which I've also seen and what we're also trying to

do with our company is this idea of like you know break away the boundaries of these different roles.

Um I think that in the past like we've gotten really good at these playbooks of look we have all these different roles right you even started by being like hey who who here is an engineer and a designer and a marketer and I think that

these roles have developed for good reasons um because you know there's very rich disciplines and crafts in each of these right so the things that you learn

as a designer around typography and usability and like patterns and interactions and like form factors and as an engineer here you're learning, you

know, code and architectures and systems and and so I think we create these because we know that there's like a really deep amount of often technical

hard skills to be able to do that really well. But what I think AI does is that

well. But what I think AI does is that it sort of makes it much easier for all of us to get into these other territories. Now again, it's not like

territories. Now again, it's not like we're going to somehow AI is going to make us, you know, the world's best engineer on on day one. And I don't think it's quite there, but it certainly makes us far more powerful. I mean, you

know, vibe coding is is the whole idea like we, you know, people who may not have studied computer science or have spent a bunch of time learning how to code can now build something and that's

like amazing. It's amazing. Um because

like amazing. It's amazing. Um because

maybe you know your skill was really understanding user psychology and what people will, you know, the specific steps that people should go through, but you didn't know how to code it. Well,

now you can actually still build something. or if you were like a product

something. or if you were like a product manager and you're very very good at actually understanding what are the specifications of something well now you can feed that into a prompt and and then you'll be able to make something and so

I think that AI allows us to not necessarily become the best at all the stuff that we're not very good at because you know there's still our lanes and our areas of expertise but it certainly becomes makes it possible for

us to move much quicker and so I think that the biggest thing is it it is going to be much more outdated I think for us to attach attach ourselves too strongly

to the idea of like what is my specific role or maybe the identity that I thought I had um in the past and you know I've told everyone at my team u look let's just think of ourselves as

builders that's like the most general thing right we yes we have different areas of expertise and that still will continue to be true but we're really all here because we're trying to work

together to build something and if we can use these tools at our disposal proposal to help us get to building the thing that we want for customers better and faster. We should do that and we

and faster. We should do that and we shouldn't be like, "Oh, but you know, I'm I'm a designer, so I can't do code, so I you know, or there's someone else who's a marketer, so like I'm going to have to talk to them instead of just

doing the thing." Like I think we can all roll up our sleeves and get to doing a lot more. Um, and I'm very excited about that. And I think, you know, you

about that. And I think, you know, you see that in the way that uh Proplexity arranges his teams, right? there's a lot more smaller pods, right? It's not like, hey, we need like six roles and

everyone's there. It's like there's just

everyone's there. It's like there's just two people, two or three people who are kind of just building this thing end to end. And yes, so two people are still

end. And yes, so two people are still designing in in a way where they complement each other skills.

Um, but it's not like there's some rule that like you have to have one of A and two of B and like, you know, like a dash of C in order to get a product. It's

like just two people can go and if they can utilize all of these different tools and they can just create something end to end and do that then go and do that right like try to keep teams to almost

the smallest amount that you need um because it's just going to be more efficient and I think everybody will feel even that greater sense of empowerment and ownership.

I think that's great. I do think you see the blurring. I mean, um, anytime

the blurring. I mean, um, anytime anyone's like, "Oh, I can't do X cuz I'm I'm a Y," that's just it's kind of limiting, self-limiting kind of thinking anyway. And I, um, you know, I think that, so it's more like

hats that people wear instead of like you are this role. It's like, okay, I've or it's like, you know, you've got multiple skills, you know, you know, it's like, okay, like it's like, yeah, being like multiple skills and and I

think you're right that it is enabling people that don't have the craft to operate in areas where they don't have the deep craft.

Yes. And it's almost like binary like now they can do something they couldn't do before. And like so VI prototyping we

do before. And like so VI prototyping we talk about I used to always talk about like the design gap at many startups and on many teams where there's no designer.

So as much as they want to do a prototype PM might maybe they do balsamic or something.

Yeah.

Run engineer can't really do so they just don't do they don't do a product high fidelity prototype or mockup. But

now you can just you can do it right.

Okay. But I also think there's like a mindset thing too which is like sometimes if I know that like I'm in a team and you know you're a data scientist and I have a data question

even if I could do it there's some part of me that's like well that's Dan's job so let me just give him the question you know. So I think we have to rewire a

know. So I think we have to rewire a little bit of that instinct as well if that makes sense.

It does. I think it's it's it's interesting. It it really I would take

interesting. It it really I would take another angle at it which is self-sufficiency.

Yes. And and I saw this in the hackathon actually just you know what was hilarious is I'm like all right we did our PD go and a small number of people

had their PRD and they had the empty prompt box and they were like and I'm like just I'm like copy paste go you know like they just but that was a hesitation of like I don't you know like you know or then you then they did it

and they would get stuck and it's like and as we all know I've been calling it the vibe coding debug death loop you know it's like you know oh yeah I fixed it oh yeah I fixed it you're like Ah, kill me now. Right. Like you have to have persistence.

Yes.

And self-sufficiency. I mean,

yeah. And you just have to sometimes like, yes, there may be, you know, because like I'm sure you guys are in teams and there's like a certain structure and there are different people with different roles, but you just sometimes have to be the one to be like,

okay, yes, I could go to this person and do it and I know they're good at it.

They'll probably do it way faster than me and probably better quality. But in

this era, I would also learn something, right? I would maybe benefit myself by

right? I would maybe benefit myself by just taking a stab at, you know, spending a little bit of time on R&D or, you know, utilizing all the tools that you flash on the screen and just becoming more proficient. And I

guarantee you the first time it's going to be way slower.

Yeah.

And it's not going to be as you probably going to be sucky and you're like, "Oh, like it would have been so much easier if I went to Dan." uh but if you actually go at it and you spend time and you get because tools also you know all

of these AI tools we have to learn and we have to get better at and we have to like there's a skill to using the tool well eventually I think we'll just allow ourselves to be more and more powerful.

Yeah. No definitely I think yeah it makes a lot of sense. Um, cool. And then

one of the things you and I talked about was um, my mental model is like for any skill PM and we can double click on okay discovery or whatever priorization or

visual design or whatever there's like a bell curve of like a bell curve distribution like for height and weight and everything else pretty much, right?

And so the way I think about it is like AI is now somebody who like would be at the bottom of the bell curve. Now AI can just at least get them to like the bottom 25% all of a sudden. Jump jump

jump them right to there. Like yes it's not going to be the world's best code or the world's best design but it's going to be better than like the bottom 10 15% of people in that field. I don't know for sure. And I mean even if you know

for sure. And I mean even if you know because it could be very well paced.

come in and you're like, "Hey, Julie, we're going to work on this feature and you know what? I just designed this thing." And it's likely my instinct will

thing." And it's likely my instinct will be like, "Okay, Dan, that's that's not very good. Like, you should have just

very good. Like, you should have just let me hand, you know, and and that's the thing like we're just so used to that kind of dynamic and that pattern."

But what I hope like if I can catch myself in that moment, I'll be like, "Well, it's actually sure, okay, this particular example, I could be a lot better and I could help him do that. But like maybe if I can spend time with you on like how do I

help you develop the eye because the most important thing is if you develop the eye then your work that you use with the tools will become better and better.

Right. So a lot of times like I think we're limited by our understanding of what does good look like.

Yes.

Um not necess you know and and because like the eye and the hand I always think of as like they're two dimensions. is

like I could have a better articulation of what good looks like and not still be able to like you know like for example I can appreciate great painting but I might not be like great at actually

painting myself. Um but the important

painting myself. Um but the important thing I think especially if you're you know working with other people around you who are using these tools is what can you do to help them develop their

eye and I think it helps us do that because then you know if we surround ourselves with more and more capable people then our team will be able to just accomplish more together and I

think we have to off al also have faith that like look it's not a it's not like a zero sum game like it's not like there's only this much work so if like Dan gets better and takes on more and it was like less work for me to do. It's

like I could do the same thing, right? I

could be expanding my skills as a marketer, as like a researcher, as an engineer, as like, you know, an analyst.

And so I could also be and and the the more all of us become better, it's like, you know, then then yeah, we can be far more ambitious with our with our plans and what we could build together.

Yeah. And I think I think, you know, I think AI is enabling that. I think when I think through my career, I've seen some of the best people, they don't stay in their lane. They intentionally learn about the adjacent functions like

designers that learn about PM and front-end coding or and PMs that learn about designer, right? Or PM or engineers that learn about design.

So, uh, and it's interesting that the term the latest term is like super IC like, oh, it's a super back to your thing, right? I'm a super IC.

thing, right? I'm a super IC.

And I think that's great. And I think that there there it's more prevalent now. It's more empowered and enabled by

now. It's more empowered and enabled by AI but some of the strongest people in the field have always kind of been acting like super ices. I don't know.

Yeah. And it it really is it's more like a mindset shift, right? I go back to like that's why we kind of have to break the old patterns.

Like we got to be very intentional about like look, that's how we worked. Yes,

that was comfortable. That worked, but like we're in a different era. The tools

are different. The world is changing and like we kind of have to push ourselves and push everyone around us to evolve.

So I agree. I think it's going to lead to smaller teams. Because the other thing is I think it's going to lead to less management levels. I've had several people at this meetup at some of our AI events go, you know what, I'm tired of being a director at this

big company. I I just want to go be a

big company. I I just want to go be a super that super IC sounds great, Dan. I

want to be a super IC at a smaller startup. Like there people are seeking

startup. Like there people are seeking that. Yeah. Right. To to get back to

that. Yeah. Right. To to get back to that connection with building as opposed that they've lost, right? So

cool.

I think like I mean again management is like it's one of the things, right?

Because maybe you know with tools we all get better at managing ourselves and you know resolving conflicts and all of this other stuff with with everyone else.

It's like maybe the task of management doesn't have to become as all-encompassing. And I think it's

all-encompassing. And I think it's awesome for managers to like roll up their sleeves and actually get back to connecting with doing some icy work.

Cool. Well, uh AI is new, the chat UI is new. You're an expert in design. What

new. You're an expert in design. What

are you kind of seeing as like state-ofthe-art for like these AI interfaces or chat interfaces?

Yeah, I think it's still really early right now. Um, now I do think that it's

right now. Um, now I do think that it's a huge paradigm shift forward that we were able to use conversational interfaces. Like I think that that um in

interfaces. Like I think that that um in of itself is like a a really a much easier way of uh allowing people to engage with questions because the

easiest thing is like just talking and and using language which we've all learned to do since we were you know one and a half or two years old. So I think that the conversation know because

actually the interesting story like chat should be chat GPT the models were around and I was like okay cool no big deal whatever it's very technical and

then they like put it into this UI this like conversational chat UI and suddenly it blew up and so again the thing that was it wasn't like the technology suddenly dropped overnight and everyone

discovered it was like no the technology was around but the interface was really what allowed people to start to access the value of the model. And so I do

think that was a huge leap forward. And

now I think you see conversational chat faces for everything. Um, and now I almost feel like we're a little bit too much on that other side. We're like,

guys, it's not this is not the end all and be all. Like I think conversational interfaces are awesome for the things that are better expressed via words. Um,

and so, so obviously if I don't like I don't the wonderful thing is I don't have to learn some like how do I use this UI and like you know which selectors do I have to like what buttons

do I have to press to like get something to happen and I think it's perfect for something like code because code is the ultimate you know thing where you have to like learn a completely new language

in order to know how to do anything in that language and so I think about a lot of the power of what we're seeing with this technology as like the power of translation, right? It's a it's a

translation, right? It's a it's a translator. So again, if I don't know

translator. So again, if I don't know how to speak a language, you know, whether it's Python or whether it's French, then this conversational interface is like perfect. I can talk to someone who speaks my language and then

they can kind of translate it into this other format. And so I think that it

other format. And so I think that it does that really really well. But um it doesn't you know I think we see its limitations as well in things that

really aren't um easily expressed with words. So for example in design I

words. So for example in design I haven't I haven't really found um any tool that I think has really made it that easy. Now again I think it's

that easy. Now again I think it's awesome to start with something. It's

like I'm thinking about an app or I'm looking for interface but really when you're talking about visuals or images like you know you think about like how do how do designers operate? like they

operate with mood boards, you know, they operate with like visuals and like colors and texture palettes and then they put it in front of them and then they like use that to to to to inspire

them to do something. Um, and so often, you know, when you use like a tool like you'll you'll get an image, right? But

then it's very very hard to tweak a particular element of that image. And so

even if you like make an app and you're like, I love everything about it, but I just wish this button was like, you know, this like theme was purple instead of blue, it can be very hard to kind of

actually direct the interface to do that. Or if you're like, uh, but I

that. Or if you're like, uh, but I didn't really I want the button radius to be like, you know, 10 pixels instead of like two pixels. It's so annoying to have to like type make the button radius

two pixels, you know, instead of whatever. And what you want is like

whatever. And what you want is like direct manipul uh manipulation, you know. And so I think that what we really

know. And so I think that what we really want is like some instances where we're high level and we want to give like the direction and then it does something, but then we want like greater tools for refinement.

Um and sometimes we want to directly manipulate the thing that allows us to um do that. And again, I think we it's a we see that you know paradigm work a little better with code like with

cursor. Yes, you have the the the

cursor. Yes, you have the the the prompt, the text field that you type into and it creates it, but you can still go back to the code and you can edit and modify specific pieces. So, you

have that granular control as well as like the highle control. I just think that we need to see that for a lot of other um modes of uh of interacting, right? Whether with visuals, I also

right? Whether with visuals, I also think voice is going to be very very interesting. Um I think right now it

interesting. Um I think right now it it's you know you can do certain things with it but in terms of like if you think about it oftentimes many of us can

communicate much more freely and expressively um and quickly by speaking rather than even typing.

Um and I I I certainly think that the idea of like us being able to use voice and do more and and and and have things operate more efficiently is something that I'm also very excited about. I was

thinking about that as you were talking because I'm like right now you're typing. Um has anyone here used

typing. Um has anyone here used whisperflow. Yeah. Right. So there's

whisperflow. Yeah. Right. So there's

there are these and it's you to your point you can speak faster than you can type. So I've been using a lot more

type. So I've been using a lot more lately. And then the other popular tool

lately. And then the other popular tool granola. Anyone using granola here?

granola. Anyone using granola here?

Like it's re revolutionized my note takingaking now. Right. So it's it's

takingaking now. Right. So it's it's yeah. So I agree with you on that. And I

yeah. So I agree with you on that. And I

do think visual is its own modality. And

so that's why you know you can go in these vibe coding tools and say build me an app. it has to pick some default UI and if you leave it to chance it's not going to be good as we know

and so like the best people still recommend hey feed it a wireframe at least or feed it this the one thing I will say to your point is more and more of them now offer direct manipulation so you can click on

something you can click the element and then you can go or you can change the class or what it's like it's getting better but it is painful to be like yeah to be specifying CSS

test properties and HTML elements in writing, right? And saying change this

writing, right? And saying change this like it's like it's very painful. So,

it is very painful.

Yeah, that I guess there's a room for a big breakthrough there. We'll see. Cool.

I I also have a lot to say about like onboarding experiences. Um I think

onboarding experiences. Um I think because everyone's like, "Oh, it's a chat interface. We just talk to it."

chat interface. We just talk to it."

I think that a lot of um products don't do a great job of actually helping you do a better job using the tool. Like you

go to all of these even vibe coding tools and there's like this text box. is

like just write what you like what do you want to create?

And I don't think they're like you know what would make your thing turn out better if you first like start by saying what the goal is and then like you know even sort of like give us three

screenshots of like things that it should the style it should look similar to and just kind of must direct you a little bit more into how you even engage with this text box.

Yeah. No, I I agree. And I'm thinking now too like there still is no tool and I don't I think this so back to the bell curve analogy if we just look at like product management design and development and

say okay for which of those functions has has like most more of the bell curve floor been lifted I think it's engineering right it's like especially front-end coding it is text somebody back to your I love your translator

analogy someone said hey the best programming language is hum human human text like normal like text so I think that's the one where it's the highest

like maybe you get to I don't know 30%.

And then I think design is the one that's the least because you still can't there's no tool where you can go and give it a non-visual prompt and have it create a good UI that I'm aware of.

Right. Or um I think for a lot of uh designers at more established companies where it's like you can give it here's my design yes brand um assets is like

our components and just use what we have already to create things. That's very

tough. And so if it just generates a completely new thing and you're like, well, I can't really use that because I'm trying to make it fit into, you know, a more cohesive um app.

And that's the other thing coding stuff is great when you're starting from scratch and you have no constraints, no design constraints, no code constraints.

But I will say the ones on the left side, the reason they're kind of designer friendly is they actually at least two of them actually let you import your existing design system.

Yeah.

So it's kind of I think they're realizing that then the flip side we can be like what percentage of people actually have a design system?

They should, you know, you should, but you don't, right?

Or help you generate your design system.

I mean, that would be amazing.

That would be super cool.

Give it a couple screens like just generate the system and imagine if it was like this cool inspectable library and you can go and change it there and then it globally changed it everywhere.

That'll be there in a couple years hopefully. Anyway, cool. Okay, so let's

hopefully. Anyway, cool. Okay, so let's switch over to uh more of the PM world.

Um, there's a lot of debate now. There's

some, you know, controversial tweets and posts. Um, do PRD still matter these

posts. Um, do PRD still matter these days or not? I mean, how do you what is how do you define a PRD?

Yeah, it's funny because I was talking to some progress. PRDs were kind of like get go you look at the Google Trends map. They were going down. Like, we

map. They were going down. Like, we

don't do PRDs anymore. Like, you know, like when I actually started my product career here, you would like pride yourself on how long your PRD was that nobody was going to read, right? It's

like, okay, the last one was 20 pages, mine's 25 pages, right? Um, but it's become it's like the catchall of the like the semistructured information

that you feed into the chat, right? Like

who's the customer, what are the problems, what are the features, that kind of stuff, right?

Yeah. And so I go back to like the spirit of what is a good PRD is clarity about what it is that we're building and

you know what are all the use cases should take care of or what are the edge cases and what should happen in this and what like success looks like and who is it for and like what should happen if like it's just clarity you know we're

trying to figure out how to make something really really clear and I think when people say it's dying what I usually take it to mean is

the instantiation of that that is the 25page doc that gets passed around may no longer be the best way to express that. You know, maybe the best way to

that. You know, maybe the best way to express clarity is now you have to you're going to show them like a prototype of what the thing is and if everyone sees a prototype and we know exactly how it works and like

that's clarity. But then it goes back to

that's clarity. But then it goes back to like well how do you create the the PRD was dying and now it's back as a simple like a simplified version. It's a

problem basically. by code the prototype, you have to like Yeah.

Yeah. Exactly. Exactly.

Or, you know, tell the AI how to build it. And like, isn't that just the same

it. And like, isn't that just the same spirit of PRD? It's like no PRDs all the way down.

PRD? It's like no PRDs all the way down.

Like, you don't if you go down that way.

That's what I love. Somebody tweeted

like they said like, "Hey, on our team, we get away with for your PRDs. We just

we just vibe prototype." And that's the spec. I'm like, "Great. What do you type

spec. I'm like, "Great. What do you type in to get the vibe prototype?" Like,

yes, you know, there's there has to be thinking though.

thinking the thinking is still what it is.

Yes. Well, let's do a quick aside on that. So,

that. So, when we've had past panels, people said, "Is AI making us dumber or is AI making us not think as much?" And kind of the analogy was like spell checkers, like you know, the number of people that can

spell. So, I don't know. What do you

spell. So, I don't know. What do you think about is it like leading people to take shortcuts and not think as much or I think it's a mentality thing. Um I so I think if you're you're just interested

in building stuff and you're using AI to build things then you're probably thinking because you have to think about like what you're building and like did this build the thing that you like you know as long as you're using your brain

to judge and evaluate and to try and you know do a better job than what you were doing before then I think it's probably helping you because you're like still you know you're still utilizing that.

Um, I do think if it gets to a point where like I'm a student, I've got to, you know, do some work that my professor assigned me and I'm really not that into

like reading whatever Hamlet and I just want to turn in the assignment so I can get an A. Then I think that use case, you know, you already have people who weren't that engaged and maybe they did

it because they had to and they were kind of forced to in this structure because they wanted that A and now you give them this like very tempting thing and it's like well like yeah like I'm sure many of them are like well I'm just

going to you know feed them the essay that Chachet gave me and I do think that maybe people in that case maybe may lose out on an in instantiation of where they

they might have otherwise in thinking.

Got it.

Um but I think it goes back to like look we have more and more choices available to us and it is our choice if we want to use it to take on a challenging thing

and make something out of nothing and uh pursue or learn something that we didn't know and now we have excellent tools to do that. But that was true even before

do that. But that was true even before AI like that was true in the internet era. you know, it it kind of got to a

era. you know, it it kind of got to a place where I remember I learned this um after talking to and spending some time with people at Khan Academy and you know, you guys are familiar with Khan

Academy is the idea was like um we can put these really really highquality lessons about math and you know, everything from K through 12 on the internet and then everybody who has

access to internet will just have access to this high quality education. And so

more or less they did that but it hasn't really led to the outcome where now like everyone everywhere in the world is and and the main reason I remember you know one of the I think it's a product

managers there was telling me that like it led to the people who were already very invested in education doubling down you know getting more out of it right so like families that are already like

really into teaching their kids will now like sit their kids in front and like you know they get ahead in math but the people who just weren't motivated it's like didn't use it. And

so it wasn't like an access problem. I

think it's more of a motivation problem.

And I do worry about that actually. I

worry about the motivation problem. Like

I worry that, you know, as these things become more and more convenient, people find less and less reason to maybe be motivated to go and do a hard thing for

themselves. And I wrote an essay about

themselves. And I wrote an essay about this, but I I kind of like and the the title of the essay was like our souls need proof of work because I fundamentally do believe that if we want

to feel, you know, satisfied or fulfilled, we have to do some hard things. Like we have to push ourselves

things. Like we have to push ourselves just for us, right? Not even for any external reward, money, whatever. It's

just for our own satisfaction and fulfillment and meaning. Like we have to put ourselves out there. We have to take some risks. we have to do things that we

some risks. we have to do things that we might fail in. And it's only by doing that that I think you learn your own strength and you learn what you're capable of and you become proud of

yourself and you, you know, you kind of become the person that you're meant to be. But I think if you always opt out of

be. But I think if you always opt out of doing it because you can and it's convenient, then I I actually think that the person that hurts the most is probably, you know, the self.

Yeah. No. And when I was asking about that, I think I was thinking motivation like basically like the the high motivation use cases you'll still think and the low motivation use cases you'll take the shortcut I guess, right? Yeah.

All right. Well, that kind of leads also to a good topic of of trust with AI, right? So I know actually along those

right? So I know actually along those lines, what there's that new hot startup that's like helping you cheat on things clearly.

Yeah. Right. Clearly. So um so trust is obviously important. So I think I think

obviously important. So I think I think you've thought about how to build trust with AI. if you want to share some

with AI. if you want to share some advice there.

Yeah. Um it's it's interesting because as AI gets better and you know obviously we're seeing new models come out every six months or every year. Um I do think this is shifting because in the

beginning with like chatt3 you know I think it used to be wrong about a lot more stuff and people would be a lot more skeptical and then every generation has gotten smarter and things but yet it

still makes mistake. I know when chat 5 launched you know a few weeks ago people were very quick to point out that there's still like a bunch of simple math problems like if you frame in a particular way I'll still get it wrong.

Um and so you know my startup Sundial works in data and like that's a use case where like nobody wants you to be wrong like ever about data. You know if someone's like what's my ARR like they

do not want even one out of a hundred times for that to just hallucinate and give like a number that is incorrect.

Um, and so I I just feel like that's you know the context really matters in terms of what we can trust. Um, and I think in very very uh important and and um, you

know, uh, situations where like you really can't be wrong, we probably still have to resort to like using deterministic algorithms. And then there's cases in which like you know

like like for example, I think images or text are great examples because there's not like one right way to say something, right? or like one right way to show a

right? or like one right way to show a picture of a bunny. And so in those cases, a little bit of that that variation can be helpful and even like, you know, we like it because it's like more creative, it's more intriguing or

whatnot. Um, so I think we always have

whatnot. Um, so I think we always have to look at like what is the context we're in and and actually I think it's probably you know every company is doing this to some degree but it's like trying to figure out like where do we draw the

boundaries of what can the LLM technology do and what is like you know especially in software like what does traditional engineering and coding and deterministic algorithms do and and

where do we even use like machine learning which is like you know another like um probabilistic model that's like different from uh the generative models and it's like where do we draw the lines

And even with every generation of better LLM's models coming out, I think we have to redraw the lines and to to kind of figure out um how that will evolve. Um I

do think one thing that LLMs do not do a very good job of is they don't really have what I call like self-awareness.

And so it would be it would be wonderful I think if like you know you could ask it like hey tell you know what is the answer and they're like well 90% accuracy that it's this or like 99% like

I think that would be very helpful you know and if you asked it like can you reflect do you think you're right or wrong that it would actually give you what is like a you know a reasonable probabilistic

uh but it just doesn't do that like if you're like hey like this is the answer you're absolutely right and you're like are you sure and they're like it like sort of thinks you think it might be wrong. So

then it like says what? Yeah. It's like,

"Oh, no, you're right. I'm probably

wrong." But like it doesn't actually have like a a real sense of like of self, right? I

find that really annoying. And I think if it did it would it would make our lives a lot easier, but it just doesn't work like that.

I've been vibe coding and and it just simple things like it always assumes it's in the right directory, right? It's like I'm like get

directory, right? It's like I'm like get commit commit and it's like there's no repository here. I'm like, "Dude, we

repository here. I'm like, "Dude, we just did the repository five minutes ago. There of course there's a

ago. There of course there's a repository, you know, like and it's just and it's but it's so confident. It's

like I'll CD and I'll do it." And it's like and it's like, "Why don't you just I'll just put I mean, check your do pw pwd just pw check." It's like it just jumps off the cliff without even like

just check, right? Like I'm like how hard can it be to just be like before you do something just check, you know?

And then it also after the fact, okay, this the bug has been fixed. Everything

should be working just fine. just fine.

Yeah.

Yeah. I want It's like I picture this like intern person like who's super super chipper pleasing but kind of competent but 10% of the time incompetent. Right. It's like

incompetent. Right. It's like

see that's what I was like I would wouldn't even mind all the incomp as long as you're self-aware of Yeah. Exactly. Yeah. Yeah. Yeah. Yeah.

Yeah. Exactly. Yeah. Yeah. Yeah. Yeah.

Yeah. Yeah. Yeah. Yeah. Yeah. Anyway,

that Yeah. So, and I think you brought up a good thing which is a good point which is in any given different product category or market category

the level of tolerable risk varies and andor level of required accuracy varies and so I think that's that because of the fundamental probabilistic nature of

these tools you know basically yeah you want a temperature zero on some of those things you don't want any variability on some of these things so I think there are certain verticals like medicine,

maybe insurance or finance. Like can you I think somebody had tried to have one manage his portfolio and like it lost all the money in like a week or two, right?

Cuz all it takes is one one boo boo and you're like oops, right? So I think that in certain industries you're going to see much later adoption and other industries you're I don't know just based on that.

I think that there are design ways to solve it like for example one of the things so our product sundial is trying to automate data analysis. So I know there wasn't that many people who were

data scientists in the audience but um the idea like I sometimes describe it as like it's cursor for data analysis because what we're trying to do is um

you know take raw data and then give you an actual story or report you know just like a really good data scientist is able to like sift through all the noise and find like what is the thing to pay

attention to and how like what's the story of what's happening like with you know revenue went down why is that right and like kind of construct a story.

We're trying to do that in an automated way. Um, and the thing is though, we

way. Um, and the thing is though, we realized like right like very early on that like okay, well, if we're going to pull any numbers again, it's got to be right. So, we're just going to do that

right. So, we're just going to do that with like it's going to be our tool, right? It's going to be just like our

right? It's going to be just like our tool calls and we're going to write an algorithm, we're also just going to execute the algorithm in, you know, the same way every single time.

Um, but what the AI does really well is it kind of can create the narrative in the story, right? So back to like it's good at text

right? So back to like it's good at text and it's much better because we actually started this company before Genai was a big thing and we actually tried to deterministically write these headlines,

you know, and it was like if this then right that and then if and then it was just immediately became clear to us that this was like an like just English the language is like not solvable and deterministic

like an infinite choose your own adventure in there.

Exactly. Um and and so when the LM technology came, we're like, "Oh my gosh, like we were looking for this for for that particular use case of like writing the report and the narrative. Um

but we're also using it internally for a lot of things where you know we have like human oversight. So I think that AI can do that as well, right? It can do a first pass at a lot of stuff. Um and

then it can ask a person to like can you check this and like does that seem right or like you know and and just do a pass and then a person can still you know put their stamp on it because the the way

that I see it like if you look at the role that data analysts or data scientists play in organization often they are who the rest of the organization trusts to interpret the

data right so we're again we're not all looking at the data ourselves often you know if you're a product manager or you're like a a business lead or whatnot, like you're just trusting that this person when they said that, you

know, revenue is down because we had like fewer subscribers last month, you just kind of trust them. Um and

you you just got to trust them every time.

Yeah.

Uh and and so uh and so we often think like, okay, well probably our product still needs to earn the stamp of approval of the analyst at the company.

like the analyst will probably have to bless our product and say, "Hey, Sundal's awesome, you know, Dan, you should go use it." And then you'll use it, right? So, like we have to make it

it, right? So, like we have to make it easy for that person to um evaluate the work and make sure that we there's transparency in how things are being

calculated so that they can feel good about it, so that they can, you know, recommend it to other people so that other people will trust whatever answer is coming out of that. And so, I just think that I mean, the world operates on

trust. you know, we very rarely are the

trust. you know, we very rarely are the ones who get to examine everything um from a first principles lens. Like we

just sort of believe what people around us say and obviously we evaluate whether we trust that person. Um but I think that that's kind of similar to if you know if you guys are all building

products with AI like I think that's one of the lenses that we can put on to like what will it be like for us to try and create products that humans can trust.

And I think I mean if people that are versed in probabistic thinking your idea of give me a confidence interval is just like that's just common sense. Don't

assume it's like black or white. Yes,

this is the right answer. Like give me some confidence interval. I think that makes a lot of sense. Although you have identified a use case where hallucinations are a feature not a bug.

If I do miss my revenue OKRs I wanted to make up a funny story good story that gets me off the hook for why that happened. That was

happened. That was Please make the room laugh.

Yeah. Yeah.

While you're going through the review process. So there you go. Um, and now

process. So there you go. Um, and now we're getting and I do think that they, you know, a lot of people, yes, they're using AI, but they they call it human in the loop. They're still keeping a human

the loop. They're still keeping a human in the loop so that like, you know, it's not just the fire and forget whatever the AI says, right?

Yeah.

So, but yeah, it could still make humans lives much easier.

Yeah. Oh, totally. And that gets to maybe the last point we'll cover before we go to Q&A, which is as this progresses, what is what is AI and machines going to do? like what's what what's going to be left for humans to

do, right? A lot of people think that

do, right? A lot of people think that we're not going to be doing much of anything. So, what are what are your

anything. So, what are what are your thoughts on that?

I think that is a that's a really big question and uh you know, I look at, for example, the story of um what's it called? Was it

AlphaGo or Alpha? Uh there so you know, it was like 1997 or 98 maybe. Actually,

I can't remember. I'm really bad with dates, but there was a year where Magnus Carlson played Oh.

um you know, IBM's Yeah. Yeah. machine and lost and he's

Yeah. Yeah. machine and lost and he's like devastated. Everyone's like, "Oh my

like devastated. Everyone's like, "Oh my god, you know, like humans are not as good at chess. Like nobody will ever play chess again because we always thought that like, you know, we're the best at it."

And then it turns out like chess is more popular than ever, right? I think like there was a Queen's Gambit. I think it did. But like

Queen's Gambit. I think it did. But like

chess.com like published it and it's like now there's like hundreds of millions of users. So it's like it's become far more popular than it ever was. And we're all fine. Like we all

was. And we're all fine. Like we all still love playing chess and doing things even if like a machine can do it better than us. And I just think a lot of things like going back to the you

know our souls need proof of work. Like

a lot of it is just for us like we can delight we I can still feel proud if I play you in chess and I beat you right.

I can still be proud if like I get better and better and I can take on better opponents. Like even if you know

better opponents. Like even if you know I know that there's clearly humans but also machines that's better than us in the world. And

so I I think that that's like one thing that for us to keep in mind is that sure there might be machines that will do the thing better, but we're the ones who get

to take pleasure out of sometimes taking on these challenges for ourselves and that there will always be room for us to do that. Um, and I think that's, you

do that. Um, and I think that's, you know, we saw that even like marathon running is increasing like every, you know, kind of extreme type of sport or or like kind of like people are just

doing more because we have more leisure time and a lot of what people want to do in those leisure times is like go and do something hard and get better at that thing that's hard. And I think for many of us who are blessed to have

interesting jobs that we like like the job has been you know a forum for us to go and do that hard thing and to get better at a particular skill and to derive some satisfaction and pleasure

and meaning from doing that. I think

that'll continue to be the case. Um I

think the second thing is that no matter how good AI gets like I think you can of course in the best case scenario okay you have like super intelligence and you can give it a problem uh you know like

okay we don't want anyone else to die from cancer we don't want you know any more car accidents we don't want this and maybe they can just figure it out right they can go out and they can execute they can do it all and we don't

have that problem I still think it's a big question like what are our problems like what does a good world look like, you know, like the questions of values are always human questions.

You need a PRD for that.

Yeah. Like you really do like it's hard.

It's like even if you know um you go go out there and you're like for example like you you're running these vibe coding workshops like people have to figure out like what do they want to build?

Right. Right. No. Yeah.

Yeah.

And like it's you you you really can't outsource like some of the most fundamental questions which is like what do you want to do?

What does a good life look like? what

does a good society look like for us?

Like um what are the trade-offs that we're willing to make for one value versus another? And I think those are

versus another? And I think those are actually the things that are most core and most human for all of us. And I

don't really see how unless the super intelligence, you know, it just enslaves us and then we become like in the matrix. Like I still feel like we're

matrix. Like I still feel like we're going to have to grapple with these questions.

Well, even in the Matrix, you have your goal that you don't know. So you still have goals. I mean, they they just made

have goals. I mean, they they just made it so that, you know, we still felt like we had free will. It's going to be so smart.

free will. It's going to be so smart.

It'll it'll trick us like that. It won't

let us know that we're enslaved, right?

I mean, we might already, you know, we might already be in a simulation living in a simulation. Exactly. Cool.

Awesome. Well, thank you very much, Julia. Great to hear your thoughts.

Julia. Great to hear your thoughts.

Please join me in giving a welcome here.

Thank you.

And then we'll switch to Q&A. So, the

way it works here, because we are recording, you can't ask a question unless you have a mic. So, we have mic runners. So, hold up your hand if you

runners. So, hold up your hand if you would like to ask Julie a question. I

will also look at Slido and try to look on Slack while we're here. There you go, George.

I'll give you here, Julia. Thank you.

There you go.

Real quick. Hello. Okay.

Hi.

So, what have you learned about managing yourself through managing others? Um

uh in recent years I have been uh working at a startup and being an entrepreneur and growing like in a very zero to one phase and I think that it's

it was very different for me in term um compared to working at meta where I was before and I think one of the biggest things is it brought me more into this question of like what should the culture

of our company be and I always find that culture is like a reflection of the founders personalities or what they believe and I know Brian Chesy has talked a lot about like founder mode and like I've thought a lot about like what

does that mean and I what I take founder mode to mean is like it's the values of the founder when they're at their best.

So again, it's not always how they behave. It's not like what they say

behave. It's not like what they say goes. It's if I think about what you

goes. It's if I think about what you know, my co-founder Chandre and I like what are we like at our best that is what I would love the culture of the company to be. And then I had to ask

myself, well, what does what is that like? What is it? What am I at my best?

like? What is it? What am I at my best?

And like took a lot of, you know, like 360 feedback. And I think whenever

360 feedback. And I think whenever you're running a company, you get a lot more feedback from everyone in like every single direction. And so that was I think very very helpful for me to try

and crystallize what it is that I really cared about. And I realize I'm a person

cared about. And I realize I'm a person who cares a lot more about the the way things are done and the process of things. Like I I care about outcomes,

things. Like I I care about outcomes, but I almost feel like I maybe orient a little bit more towards like how we do things because I think that that I think

I implicitly believe that that will uh influence how what the long-term outcomes will be. So I realize that that I care a lot about that. I think the second thing that I learned about myself

in terms of managing a company is this idea of conviction. In fact, I just wrote an essay about that which I published today, which is I just I feel like the only thing that really kills

startups is when you lose conviction, right? Nothing else is actually a real

right? Nothing else is actually a real killer. like if you get rejected or if a

killer. like if you get rejected or if a customer says your product sucks or if somebody quits on you or maybe you weren't able to get that money uh or a

VC says no like none of those are actual killers in of themselves. What they do do is sometimes they can chip away at your belief that this thing is possible because whenever you start something you

always start by saying of course I could do it right I can do it like could be hard you know I'm going to probably face some road maps but I think I could do it is if you didn't believe you could do it you wouldn't have started and so for me

the process of like actually going and doing this startup is like I started with a lot of conviction my co-founder and I and along the way you know all these little things test us right Every

rejection tests us, you know, customer being like, "This product's not as good as I thought it was as you guys made it out to sound." Like it it like each of these is like a a thing that might trip

a little bit off that conviction, but then we have I have to kind of play this game of like how do I keep the conviction alive, you know? how and it's kind of like a mind game like every day

like what is it that I can believe and how much can I like you know keep that fire of belief going because if it keeps on burning forever then I think we can

keep doing this forever and eventually I believe that it'll lead somewhere um but the true killer the thing that I'm always most concerned about is like if conviction dies and so I think a lot of

it is just like I've learned a lot about like what is it that makes me believe yeah Since you brought up founder mode, I just want to chime in on that. So,

it's funny because it's like two words, founder mode. This is what always

founder mode. This is what always happens whether it's MDP or I was someone's getting with me on vibe coding on LinkedIn the other day like people read into that. So, it's like founder mode. It means you're God. You can come

mode. It means you're God. You can come in and do whatever you want. Like you

can blow things up. Like my take on founder mode is it's meant to be the opposite of what h naturally happens as company grows and get big and bureaucratic that the people at the top lose touch with what's going on which is exactly what we were talking about

earlier about like managers getting more hands-on rolling up their sleeves. So

that's what I view it as. And I've

worked in companies where even though people are senior leaders they are still in touch with the details or they're in a meeting and they can ask the probing questions to go down. Right.

So I don't know that's my take on it. So

yeah. All right.

Hi, my name is Jude and my question for you is with AIPM becoming a more prevalent job um especially in this age, what advice would you have for young professionals um with the traditional AI

background with PM?

Do you mean professionals who are PMs?

Uh so basically with like younger like professionals such as myself who have background like AI and PM but not just roles that are actually like artificial intelligence product managers

artificial so it's a question more like how do we stay relevant?

Well how do like younger professionals adapt? What advice would you share for

adapt? What advice would you share for them?

H I mean I think similar to what I said before um I think if we think of oursel as like I'm a builder. I just want to build some cool stuff. like what is getting in the way of me being able to

realize the vision of things I want to build. Um, and then you know if like the

build. Um, and then you know if like the blocker is oh I don't think I have like access like I don't really know how to get great user experience then like okay cool like work on that right work on

that so it blocks you from being able or if your your thing is like I don't know how to realize this in in production code then figure out like if there's better tools or how can you can you know

learn or and again you may still have to like pair with people still humans because I don't think AI is good enough yet at a lot of these things to fully replace humans. Um, but if your goal is

replace humans. Um, but if your goal is like often times a product management goal is just you want to deliver the thing, right? You want to ship the thing

thing, right? You want to ship the thing and you want the thing to have like an outcome for humans. Um, the product management role is almost like the amalgamation of just trying to make sure everyone else does their thing. Um, but

if everyone else doing their thing means you just get better at using tools to take on all of these other skill sets, I think it's going to make you far more powerful in the goal of reaching the

thing that you want to build.

Thank you. All right, we have a top voted question from Slido. Six votes

coming in. How does this technology change product discovery?

Product discovery as in how do people dis like how do customers learn about your product? No, like how do you as a

your product? No, like how do you as a product team do discovery on your customers or needs in the market? Like

how does that change or get better?

I see. I see. Yeah. Um, well, there's actually a lot of tools I think in that space as well, like tools that help make it much easier to do a user interview or you do a user interview and it

automatically analyzes it or you give it like a bunch of footage of users using the thing and it like learns about what it is. Like I'm actually quite excited

it is. Like I'm actually quite excited about about this because to me it's like you know I'm kind of a data geek so it's like getting easier and easier for all of us to have more access to data and I

think in a data world you know I I think of data very very holistically right so I think of like qualitative data like what do users say and when you interview them what did they tell you they like

and when you ask them to rate your app and you know how many stars they give like that's all more qualitative um but that's data and then there's stuff that like, well, what do they click on? How many minutes

are they spending on each part of your app or your site? Like, that's like quantitative data, but it's all data.

And the us the difference in the past used to be that a lot of qualitative data was hard to collect at scale, right? Because you'd have to talk to the

right? Because you'd have to talk to the user and a, you know, a researcher would have to do the interview and then um and it takes a long time to do like 25 interviews and then they would have to like, you know, use their judgment to

try and figure out like what Yeah. what

is the the key things that are coming and now I think we can do that at scale with um with AI. So I actually think we should get a whole, you know, much wider

range of like qualitative signals, you know, to the point where like over time, you know, if you just like maybe there's a camera and like somebody you're watching someone, you can like study where their eye movements are, what

they're looking at or like what their expression is or, you know, and that could just be many more rich dimensions of data which helps us get a better idea of what it is that people care about.

Yeah, I think they're seeing some I think it's still early days, but like imagine to your point if there's a transcription of all the user interviews and you can just go in and say, "Oh, this one interview complained about, you know, this page and then you put type

that in and it pulls up all the clips that have that phrase in it and you can quickly pull those together into like a highlight reel to share with other people or something like that's I think what like dovetail and some other tools are making possible now." Yeah,

it also reminds me of another thing which our company is also very very interested in which is you know now that we have conversational interfaces.

Conversational interfaces are actually quite hard to analyze, right? It used to be you know you would know what like if you go someone goes to Google you'd be like oh they want to they want an image cuz they clicked on Google images or

they want maps cuz they clicked on maps or they want shopping cuz they clicked.

Right now we can't measure clicks anymore. you talk to Chachi BT and you

anymore. you talk to Chachi BT and you just ask it a different question and how do we know if you're talking about shopping or images or like directions or whatever it is and so it's like a new

kind of analysis and of course we have to use AI you know first to try and bucket and and sort but then to to actually understand because one of the common questions I'm sure if I worked at

on chatbt is like what use cases are growing totally that's why they have the thumbs up thumbs down now we know why they have it because they have one of their few the rough feed back mechanism. Yeah.

Interesting.

Yeah.

Yeah. I think something you said really resonated which is onboarding and user training in this world. Um and like you said chat is actually it's just the beginning. It's not sufficient. Any best

beginning. It's not sufficient. Any best

practices you have for that and I asked because I I work at the startup AI where we're kind of building one of the things products we have is like a cursor for video editing.

Like you take an unedited video you prompt to edit it but then you have to train a user. There are certain kinds of prompts you can write to do like edit it certain way and there's a bunch of

training that goes into it but you also don't want to write a document telling you write these prompts don't write like what are some I've seen suggested prompts that's what we used to do at meta as well but like anything else you've seen

well I do think okay right now I I think a lot of people who are using AI are like early adopter audiences and so many of them just early adopters like just want to try to think like don't make me read a bunch of stuff don't make me

click through like 20 pages of onboarding, you want to try the thing.

So, it kind of makes sense why these products are like that. But I actually think that what you know what would be really cool is like you let people do that, but then after they try their first thing, you ask them like, "Hey, do

you want a little more guidance on how how to you know structure this prompt or or to figure out how to help you get the outcome that you want?" Maybe you know even on that page it's like here like

like click here to see like some guidelines on how you might do a better job of giving instruction to the AI to get you the outcome that you wanted. So

I just think that there are certain people for whom like if you say look these are the best these are the best tips and practices that we have for great experience they'll actually appreciate that and they'll learn from that and then maybe like you mentioned like

also have it in the output like is this working for you? Do you want a little bit more guidance in something?

Yes. Exactly. Or you know they try their first thing if it isn't good and they try again then in that case maybe you say okay let me give you a template like fill things out in this way and you may

get a better outcome.

I've also seen just lightweight user guides at the beginning you know it's like hey you know like maybe you walk them let's walk you through editing your first video. You know I just did it actually

video. You know I just did it actually was Granola did it. Granola, check out Granola's onboarding cuz you have to like learn how to use the mic and do it and it literally like this video of the founder. He's like, "Okay, Neil's going

founder. He's like, "Okay, Neil's going to transcribe our meeting here." So, you know what I mean? It's like a meta thing, but you take them through editing the first video and you teach them the most common ways to edit it. Then they

know those that's so that's and you don't want to be too long and heavy-handed, but lightweight. There's

some really Check out Granilla's onboarding.

Yeah. Um, and then what I would say also is what I find super useful is like piggybacking on what Julie is saying is say they do the first thing

you know that there's probably three or four common things that people want to do next that they forgot they didn't know about like just make those facets expose those facets and say would you like to now apply this filter would you

like to you know make it shorter would you like you know what I mean like take a smart guess you should be able to figure out from quality updated data.

What are the top things people want to do? And if you have little affordances

do? And if you have little affordances right to the side that let you do it, right? So I don't have to know how to

right? So I don't have to know how to type. Anyway, so thought some thoughts.

type. Anyway, so thought some thoughts.

I like to see more templates. I think

templates are are underutilized right now in uh conversational AI.

Yeah, we're putting a lot of suggested prompts, kind of detailed templates like just use and we actually seeing most people are lazy. They just pick a suggested prompt and run with it.

Um but I think like kind of just having more and more affordances like that.

Yeah.

Yes.

Hi. This regarding the PRD um prototype discussion. Um so I mean I read this and

discussion. Um so I mean I read this and people are in product managers creating prototypes and showing. Doesn't that

really cut short the discussion of what a solution should look like because then once you see the prototype people start reacting only to the prototype as opposed to really stepping back and thinking what are we trying to solve.

Yeah.

Right.

So I do think you're absolutely right. I

think there's a lot of things in a good PRD that you still want to make sure everyone's on the same page about. For

example, the prototype doesn't answer who is this for, right? What's the target audience? And

right? What's the target audience? And

so, you kind of still need to say that out loud and like, are we on the same page about that? And if people are not on the same page about that, they're just all going to evaluate the prototype differently because they've got a different bar. Because again, like if I

different bar. Because again, like if I think it's for power users and you think it's for like new users, like we're just we're you're probably going to be like, "This thing sucks." and and we're just going to be talking past each other. So,

there's certainly a lot of value to just like even before that just making sure we're all aligned on some of the bigger picture items. Right.

I I think you bring up a good point and I would say like honestly even before AI I think Figma contributed to this like when it's the thing is the easier it is to create some highfidelity prototype as the first shot

or the more people want to do that as the first shot then the less there's discussion of interaction you know information architecture navigation design patterns things like that it's just like here's

this one is like I think good teams do divergent thinking some and they go hey there's three different ways we could approach purchase, right? And they're

they spend time in a lower fidelity medium like wireframes or something. So,

and unfortunately, I think vibe coding just has accelerated that now because like not only now is it it's not a high fidelity clickable prototype, it's actually working HTML, CSS, JavaScript now, you know, and people just I in my

book I have the iceberg metaphor for for UX design. The tip of the iceberg that

UX design. The tip of the iceberg that everybody sees above the water is visual design. And so when people see it, I

design. And so when people see it, I remember I had one CEO, our designer would create these amazing, we would make these amazing HTML CSS prototypes and he would just be like, "Oh my gosh."

And he fall in love with it. He want to ship that thing right now. Right? Or

he'd be like, "What's this green? I

don't like the green that you guys use."

I'm like, "Are you serious? Who are the customers? Did we build the right

customers? Did we build the right features? Did we do the layout right?"

features? Did we do the layout right?"

Right? So

and if you take that, the role of engineering is cut short even further.

Right now, now you're coming and giving me the product and say fill up the gaps.

Right. I did all the UI just look at you know latency performance and things like that because when I used to run engineering I mean I did product management also it really used to upside engineers when people come and tell them

just do this right don't ask me any questions right but again the engine what's to say that engineer can't go and like also be the one to design the next thing you know what I mean sure yeah but I think yeah I mean that's a

second question I have is why is chat is not much more prevalent because like when I for example I go to my bank account right I still see the

old pages and everything and every time I go there I have a different intent right but it shows me the same remember what I said about riskiness the financial services people are going to be the last ones to adopt it because the

risk right they're not necessarily the first adopters of new technology waves anyway would you want a chat interface I think I do see some that have incorporated it I want because whenever I go to Wells

Fargo I have a different intent right sometimes I just want to go and see the check has been deposited that, right?

But it shows me the same page every time, right? Because, okay, it shows me

time, right? Because, okay, it shows me the front page with all the accounts I got to click through to find. But I

thought, okay, if you could ask me a question, then create dynamically create the UI, that would be much, you know, much better for me, you know.

No, I agree. I think if there's a way to just chat Wells Fargo and it's like, hey, did that thing get deposited? It

just goes, yes.

Search your data. Yeah.

So, would you want like a little paper clip in the bottom right corner?

Hey, it looks like you're trying to find a check today.

The only thing I realized is that, you know, the information, as you said, sometimes it can be in a table, sometimes can be a chart, but then you test it dated dynamically, you know, depending on what I want.

And over time, I hopefully should get really good at personalizing like if you're like, I'm a tables person because there are some people who are like that and then it'll just show you dense tables. There you go. Um, one of our

tables. There you go. Um, one of our investors was like a table person and he always just wanted like all the data in like this very dense format and then you're a chart person you know you'll get things more visual. Actually, I had one more thought about the prototype,

which is that, you know, right now we're assuming the prototype is just how the feature works, which I think is useful, but we can also get to a world in which maybe one day it's just really easy to create like um it's like a marketing video.

Yeah.

And the marketing video does a good job of talking about like who is this for, you know, and it's like maybe before you make the prototype, you make the marketing video and you get everyone on board like is this the right marketing video? Is this the way we would talk

video? Is this the way we would talk about it? Is this the the kind of

about it? Is this the the kind of audience we would be pitching it to? And

then if you agree then it's like okay then this is what it looks like.

Then you'll get the disconnect between the marketing video and the prototype like this is amazing know I think that's called Kickstarter.

Yes.

Uh all right cool. So the next top question from our friends on Slido it's moving up is can you provide examples in typical enterprise workflows that would work better with AI in terms of efficiency? I think they're focused on

efficiency? I think they're focused on enterprise workflows and they're saying eg could PMs could mock prototypes to verify faster and they mean like in enterprise

companies like B2B companies I guess but like what specific workflows any uh like product related workflows like they're mentioning prototyping for example so to increase efficiency

I think everything we talked about discovery yeah is like all relevant whether it's in a B2B or B to see environment.

Sounds like you should come to our vibe coding hackathon September 17th and get your hands dirty.

All right, cool. All right, let's go.

Hello. I'm going to stand up to honor you.

All right, so Julie, my name is Genevieve. I have

been reading The Looking Glass for years and it has extremely influenced my own development as a design leader and I'm now a coach and I use all of the learnings from the making of a manager

to coach people how to be managers. So,

thank you for that.

Thank you.

Um, and it did it made me realize I have a question for you. So, one of the essays I loved the most from the looking glass, I can't remember the title, but it was about the importance of being a

follow-through person. Not just an idea

follow-through person. Not just an idea person, but a follow-through person. And

this went straight into my brain. And I

realized this so briefly the essay for everyone to understand is of course you can have a great idea. Almost anyone

could have a great idea, but in a company to make outcomes occur, you have to be a follow-through person. And it's

not always about having the best idea, it's about having the best followth through. And it changed my life when I

through. And it changed my life when I read that essay. So I was curious while you're talking about the influence of AI and product development, how does your attitude about follow through and the

importance of being a follow-through person carry into this era we're in now.

Thank you.

That's a really good question. Thank you

so much, by the way. It just it means a lot to me to hear that it was helpful for you cuz you know I write I write these things and they're kind of like letters to myself like what I need to hear. Um and I always get so delighted

hear. Um and I always get so delighted when other people find it helpful. I'm

like I'm not alone in this world. Um I

think that you know one of the the things that your question made me think about is is already this idea that like it's so easy to express the first version of the idea with vibe coding and

it's actually really difficult to extend that to the rest of the product development process to get something into productionization. Right? So it's

into productionization. Right? So it's

almost like we made the first part of it easier. We made the expression of ideas

easier. We made the expression of ideas easier. we can get very very excited

easier. we can get very very excited about them, but actually making the thing, you know, work for the outcome that you were going for, which is what I think, you know, we're all trying to do as builders, right? We're like, it's great to build the thing. It's great to

have the idea. It's great to, you know, have a a mock or a live version of it working, but it's got to go out there in the world and make some impact. And if

it doesn't make any impact, then then sure, it was fun, but like we didn't really do what we set out to do. And I

actually think it's like it's very like I have so many projects I've started in the vibe coding era and it hasn't actually gone anywhere. You know, I I thought I was going to do this and I was going to build this internal thing for

the for the for for my team and it was going to be part of the internal process and I got like you know 70% there and I didn't actually carry that through to the out to the impact. And so I actually

think this message of well what is the impact we're looking for and like can we still measure it based on what actually happened continues to be a very very important thing for us because it's

almost like easier to feel like you're making progress. And it reminds me of

making progress. And it reminds me of this poster we used to have at Facebook which is my favorite posters. It's of a rocking horse and the words say don't

mistake motion for progress.

I think that a lot of this shiny AI stuff can make us feel very excited because we we you know a lot of motion and like look we're all looking at this and it's awesome and even like you know

everyone's got even more ideas but are we actually taking it all the way through and impacting customers right that's what I think it means to be a builder here in this era

and I and I think that the you know first generation it's easy to build from scratch like you know blue sky green field but increasingly people realizing this thing of well there's a whole other

use case if I've got a thousand lines of code and I've got a design system so I can't just vibe code whatever I want I need to live within these constraints that they're starting to address that more and more I actually for the first

time was speaking with a client and their PMS have get access and have a local dev and they can fix but now they can't they can they can't deploy it but

they can push a pull request to get reviewed like and so the interesting thing is like like little bug fixes and stuff like that they can do. The other

thing I'll say is we're at a very important inflection point right now with the LLMs. If you use some LLMs, you will get in the debug death loop and you will never get out. And then I switch

over to another one. I won't say and it's like I feel like I just got upgraded to a Mercedes-Benz. I'm like,

"Oh my gosh." And it just solves it the first time. So like the models do matter

first time. So like the models do matter and I think we're at this inflection point. like GPT5 specifically was

point. like GPT5 specifically was designed to be better at coding than previous versions to better compete with cloud.

That was very very deliberate. In fact,

one of the early review videos of it, it was spitting out like HTML comments and text output. You're like, uh, so anyway,

text output. You're like, uh, so anyway, so I think we're in interesting inflection point. Everybody sees they

inflection point. Everybody sees they they want the rubber to meet the road.

They don't just want a bunch of hype.

Yeah.

And and and people want to drive it forward. Um and I think you do see some

forward. Um and I think you do see some on the farther end some hardcore engineers really using cloud code really using cursor with production code. So

it's like I almost now distinguish between vibe prototyping as the left side and vibe coding is the right side and like you know so I also think though when everyone it make we lower the bar everyone makes

more things it is still harder like attention is still quite finite and so to capture attention and to win over customer hearts and minds becomes an it

still becomes it's still a competitive game because now if everyone has access to these tools and everyone can make things better and faster It just the comp the competition is still there and I think we still have to hold ourselves

like if our goal is to win in a marketplace and be better than our competitors and win over those customer hearts and minds and dollars that I'm not sure that that is getting any easier. In fact, it might be getting

easier. In fact, it might be getting harder in some ways. Um and we're at least more competitive.

Cool. Well, I think that's a great point to end on. Why don't we end it there and we can just do individual Q&A with folks. Okay.

folks. Okay.

Cool.

All right. Thanks.

Thanks everyone.