#89: The hidden role of rituals in Duolingo's product culture | Nickey Skarstad (Duolingo)

Alrighty, we are live. Nikki, thank you so much for joining us.

>> You got it. Thanks for having me.

>> You uh you are one of my favorite people who I do not hear from enough um in the product leadership space because you are heads down doing a lot of very important uh work that a lot of us unfortunately

can't see in the public uh at Dualingo.

But I'm a big fan of what I do see and uh you joined for a really fun session I hosted earlier this this year um around what PM candidates are doing to really

impress companies uh these days. So

beyond the hiring market, Mark and I just wanted to take some time and go deep with you on your thinking about the actual role of uh of AI and evolving the

product management space and as a product leader at a very hot company right now that is becoming you know well known as being an AI forward and AI first kind of company the kinds of things you think about these days and

where you know what your impressions are. So maybe where we can start the

are. So maybe where we can start the conversation is you mentioned that in addition to doing just like your regular quoteunquote job, you also have to carve

out very intentional time to stay on top of the latest evolutions with AI so that you can help your team incorporate that into like the way you build, the way you work. So maybe the first question I have

work. So maybe the first question I have for you is just like what have you found to work well for you now in staying on top of it? like how what's your what are your rituals? How do you how do you try

your rituals? How do you how do you try to make sense of it right now?

>> Yeah. So, I think one of the most important things for me has just been making sure to carve out time to be following along. I think things are

following along. I think things are changing so fast right now, which is exciting. Like it's a really fun moment,

exciting. Like it's a really fun moment, I think, to be a builder. But I also think there's pressure, especially as a leader inside of an organization to make sure that like you are paying attention to what's happening and that you can be guiding your team and helping your team

understand what's happening because again things are changing so fast. And

so what I've been doing um is really just making sure that I have a few different newsletters or podcasts that I'm listening to every week. Um and then I'm also just trying to carve out time in my actual workday to like make sure

I'm paying attention to and reading up on what's happening. Um so some people I follow that I really like uh their content, Peter Yang, somebody we all know. I think he does a really good job

know. I think he does a really good job of just like staying on the top of like what's happening and he brings a lot of good change makers into his podcast.

also loved Lenny's newsletter of course um just to kind of hear especially like the big leaders that are coming out of some of the big you know AI houses just hear so what they're thinking about um

and then also just you know surprisingly I think is maybe because it's my network is really good is it just LinkedIn is another place where just checking in there I'm sort of seeing like what's new and what's happening every single day also inside of Dualingo we have a

handful of folks who are you know it's their job actually to be keeping everyone updated and so I'm paying attention to that too and making sure I've got my ear to the internally as well. But I think that's

well. But I think that's >> is that like a like a newsletter that someone internally is curating almost like here's the latest, here's what you should know.

>> Exactly. And then we also have a Slack channel where people are just posting when things are happening in the Slack channel and that's sort of like organically I think somebody just created it. Um and it's a good place to

created it. Um and it's a good place to pay attention because as things are dropping like that they'll get posted in there pretty immediately and so you can kind of keep keep an eye on that as well. Um, but I was saying earlier that

well. Um, but I was saying earlier that I think is very true is this has become like I wouldn't quite call it a part-time job. Like it's not that many

part-time job. Like it's not that many hours a week, but like several hours a week probably I'm making sure that I'm just paying attention to what's going on. And the good news is I'm building

on. And the good news is I'm building for my job. And so some of that were of trying things and doing it directly on the content that I'm work or like the subject that I'm working on inside of Dualingo and then have trying to trying

to carve out time to do some side projects which is easier said than done when you work an operator operator schedule like mine because I'm definitely busy but I'm trying.

>> So it sounds like you're spending Yeah.

I call it like two to four hours a week give or take on just keeping up with like the the news and that's either reading newsletters or podcasts or

checking out your internal Slack. I'm

curious like as you're reading all this or you know consuming all these like media diet that you have for AI like what are you looking for and and what are some signals that maybe like you're like maybe I should take action here or

maybe this this is a tool that's worth um like playing with versus like I'm just going to keep it in the back of my head and I don't have to take action.

>> Yeah, I mean I've definitely been I would say more focused on this the products that are coming out of some of the big AI houses. Uh just this morning somebody posted in our product management channel that they were playing with Sora. And so I was playing

with Sora this morning and it sounds cerebral. You're like what's she

cerebral. You're like what's she creating? Very important things for

creating? Very important things for Duolingo. I literally took a picture of

Duolingo. I literally took a picture of my dog and I made a video of my dog that was funny who she's scared of squirrels and it was a whole [laughter] thing for my family. And so it doesn't it doesn't

my family. And so it doesn't it doesn't have to be cerebral honestly like you can have fun when you're doing some of these things too. And so I think walking that balance beam is important for my own sanity.

>> So it also makes it fun too.

>> Totally. Yeah. So, it sounds like you're not immediately trying to connect like the part of you that's just trying to stay in touch with a technology and stay on top of it >> doesn't feel like you have to

immediately connect the dots to some some use case at work or for something you're doing.

>> Uh it's just could be something personal. And then have there been any

personal. And then have there been any patterns in the kinds of those tinkering that led you to then have the bigger aha moment which is like okay I need to >> this is something we should incorporate into like either something we're

building or how we work like what what what does it take for something to kind of graduate to that >> threshold. That's a good question. I

>> threshold. That's a good question. I

think the biggest thing that was important for me is just like using chat GBT regularly and using Claude regularly. I like both of those tools. I

regularly. I like both of those tools. I

use them both pretty interchangeably and just really teaching myself how to prompt well to make sure I can get good outputs. Um the product that I'm product

outputs. Um the product that I'm product space I'm working on inside of Duolingo is Duolingo math and so a big part of the challenge that we have is we're actually using content or basically models to generate math content. And so

being able to prompt really good and have good evaluators is such an important part of the work that myself and the team is doing that to not be an expert on how to do that myself like would not have been helpful for anyone.

Um, and the good news again is you don't you can do that in a in a straightforward way, which is just like playing with the tool, understanding the edges of how prompting really helps outputs and then like passing along things that are really helpful so the

team can see it too. Um, and the good news is too it's like, you know, I'm I'm trying to set an example that I'm learning all the time, but the team is doing the same thing too. Um, and so that they're trying cool things and they're sharing. And so I think there's

they're sharing. And so I think there's this like really awesome collaborative energy and ecosystem that we've created.

So everyone's kind of learning from each other. Um, and everyone's not scared to

other. Um, and everyone's not scared to try things. I think especially with AI,

try things. I think especially with AI, like you what you start creating typically is not good.

>> Um, it gets good over time. And so you got to create a really you have to create a culture where you're not wrong and you're not doing things that are bad. And just because you know that when

bad. And just because you know that when the team is learning and practicing and doing it more, it's going to get better over time and you can kind of bet on that.

>> Have there been any specific rituals?

Sorry, Mark. I'm just curious.

Nick, you mentioned like sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sh sharing with each other what you're all learning. Are there any specific rituals like showand tells or any specific things that you found to be most effective for kind of creating that internal sharing?

>> Yeah, I mean I think it's just posting in Slack sounds dumb, but it's just a, you know, it's an asynchronous moment.

It's easy to share and update and the team sees it and then they can do the same thing and then everyone's watching that. So that's a good ritual. Um, we

that. So that's a good ritual. Um, we

have demos every other week on Fridays and it's meant to be again there we have this concept of hot trash inside of Dualingo where it's it's okay if it's not good like especially with a demo or something like that. It doesn't have to

be perfect. We're not saying we're going

be perfect. We're not saying we're going to ship this. We're saying like hey look at this thing I tried or look what's new and we're kind of playing with and 50% of the time like you definitely would not want to ship that or you couldn't

ship it anytime soon. But then 50% of the time you're like, "Oh, I can totally see how that's going to get good soon and we're going to be able to get that into the product experience and into people's hands." And I think that gives

people's hands." And I think that gives everybody energy because then everyone sees what everyone else is doing. And

then there's other things that I think are more typically these are like more technically led, less productled. Um,

but just having good forums where people can kind of come, especially around generation, and just say, "Hey, here's what here's what I've learned. I've

created a generation pipeline. Here is

what is working and what's not working."

And then the team, other people can come get advice, see what's working and not working. And then that just kind of

working. And then that just kind of uplevels what everyone's creating. Um,

and then that stuff gets shared back in Slack typically as well, so everybody sees it. So it's just kind of like

sees it. So it's just kind of like trying things, experimenting, and then as much as possible sharing those things back with the team. Uh, so everybody's learning from each other.

>> Yeah. Um

I'm curious to how has you know maybe looking back maybe like a year or or 9 months from from today like has the way

your team builds product you know all the way from like the discovery to shipping fundamentally change at all with these new tools?

>> Yes. Let's see. Yes, I think so because we're building things in different ways now than we were before. And so, not in every case, but we're using AI a lot in a lot of ways to help us scale the things that we're working on. And so,

that is just really new. And it's also just gotten a lot better recently. And

so, we've definitely been doing that for like the past year, but I would say like in the last six months has really ramped up. And now we can actually ship stuff

up. And now we can actually ship stuff um and feel good about it, which is important. in terms of like the core

important. in terms of like the core like product ritual like it had I don't think it's changed a ton at least in how we've been building because we were kind of doing we were sort of innovating already and like that was already

happening and so we're still talking to users we're still trying to move really fast um Duolingo has really great sort of product principles and company principles one of the things we talk about a lot is clock speed in the green

machine clock speed is just moving really quickly and so like AI I think has helped us do that even more quickly which is good um and then the green machine is just trying a bunch of little things, doing a ton of tests and

iteration. And so we're, you know, just

iteration. And so we're, you know, just incorporating AI into our process along those lines as well. And so I think like theoretically like the work, the day-to-day work is a little different,

but like the way that we've been we're building and what we care about like hasn't changed. Um, especially, you

hasn't changed. Um, especially, you know, Duolingo is in so many users pockets all over the world. And so we're not going to ship things unless we think the quality meets a bar that we are

proud of. Um, and that is for sure has

proud of. Um, and that is for sure has not changed, which is challenging with AI because especially when the models are still new and we're all still learning, like not everything you're going to build using AI is going to be able to be shipped at this stage. Um,

and so I think actually this is good advice for any product builders out there or anyone who leads an organization is keep your product quality principles really high. A

quality bar should stay high. Um, it's

important for teams. It'll actually push everyone to figure out how to use AI in ways that where things can still ship.

And I think quality can easily ship if you're or slip if you're not careful.

And so you really got to kind of keep a laser focus on it.

>> And as a product leader, like how like where where are you putting those checkpoints to make sure that the you know the product quality doesn't go down? Is it like every step along the

down? Is it like every step along the process or is it mainly just like product review like before you you turn that flag on? Like where where you've Yeah.

>> Yeah. So we have I would say like a handful of different rituals that kind of all have a quality element. our teams

inside of math. Uh we have these c cross functional meetings that hap happen every week where it's typically the team leaders or the team leads uh that come together and quality is a big part of like when you're reviewing the team's

product and what they're working on. Um

and so just really pushing on that and making sure like everybody knows what good looks like, everyone's aligned on what good looks like and everyone's sort of sprinting in that direction. Um and

that's at the team level. And then you go up through Dolingo has this famous product review process. We should link to some there's some good articles about this we could put in the show notes. Um

but uh what this is is basically like a ritual that we have that also puts a focus on quality and so you're bringing in at different product project stages and getting feedback from senior leaders

at Duolingo. And that's another way that

at Duolingo. And that's another way that we keep quality really high. And that's

a very streamlined like we we've got this down like to a tea at this point.

There's a ton of process around it.

Sounds annoying actually. I feel like when I tell people about Duolingo's product process, they're like, um, but what it does, I I think it just it's just this giant quality mechanism that

helps the entire product organization know what quality looks like because everyone's in the room together. They're

looking at the same thing and they're hearing leaders opine on if the quality is good enough.

>> And so the reason that rubs people the wrong way or that they might hear you talk about that and it sounds kind of like they have some kind of negative reaction to that. What do you think is underlying that that reaction that it feels what?

>> Honestly, I think like there is good process and there's bad process and we've all been victims of bad process in our careers at different times. Um, and

so I think sometimes people hear about things like that and they're like, "Oh god, that's going to be bad." And you have to just make sure, especially if you're putting into place a ritual like that, that you need to like keep iterating on it because it's not going to start good, but it needs to get good.

Um, and so I guess I'm guessing that's where that comes from is people are like they're they're imagining that this is a poorly run thing or it just creates a lot of extra hoops to jump through. And

I actually think we've done a good job of like meeting the teams where they're at. And so we kind of like we check in

at. And so we kind of like we check in with them at the at the points of their product process where they need some advice and some feedback and it's meant to be more focused on helping rather

than blocking things. But I definitely think, you know, I'm sure you all can remember some times where you've had bad process in jobs.

>> Yeah.

>> Yeah. I mean, it's interesting because in a way like I work at companies and I know of companies that have like a very strict product review process.

>> Yeah.

>> And a lot of times because of that um kind of what you're trading off is the speed of execution, right? Because

then it's like you're spending so much time doing work about work um and polishing everything. So I'm curious. I

polishing everything. So I'm curious. I

mean it sounds like it sounds like yes speed is very important at Dualingo but also you have this serious product review process. So what what do you

review process. So what what do you think are some how yeah how has Dualingo managed to keep this balance in a way that doesn't slow people down?

>> Yeah, I mean I think there's just been a ton of iteration on the ritual and the ritual when it started years ago I think it's been around for like way longer than I've been at Dolingo. um when it started, you know, it's very different

today because it's had to scale with a growing organization and like we have a lot more PMs than we did five, six, seven years ago. And so I think that works. And then the other thing is I

works. And then the other thing is I think everyone understands that clock speed is important. And so it's pretty quick teams only talk for like five minutes. Ideally, there's not an over

minutes. Ideally, there's not an over there's not a ton of upfront work that needs to happen because you've already made decisions and have images have basically mocks or prototypes created.

Um, and so again, like that was kind of already our process even before pre before AI. Um, and so I think that helps

before AI. Um, and so I think that helps it feel much more productive. And the

other thing that I think I've seen it do, which this is, and this has been missing at other, you know, I've worked at Etsy, Airbnb, I worked at startups, like a lot of different mostly consumer companies. This is the most effective

companies. This is the most effective ritual that I've seen. And I think that has helped at least be much more palatable inside the company because everyone sees the value it adds. But the

one thing that it does that I think is enormously helpful is it creates this like visibility mechanism where everyone understands like where the feedback that was given and then what good looks like

and that permeates like through your entire organization and I think Dolingo's product quality is so high because of a ritual like this especially as the or grows like everybody going and

kind of hearing and seeing what good looks like makes everybody better at building stuff. What I'm what I'm

building stuff. What I'm what I'm hearing is that you've established and you being the organization which you're contributing to, but the organization has created a ritual. Tell me if I'm off

here, but it sounds like it's a mechanism for scaling taste and like like judgment across the org.

>> Totally. And I think it's especially helpful for like, you know, when you're a senior leader and you've been doing this for a long time, like your product sense and your product taste typically is like pretty well honed, right? But

it's really great for junior like APMs and PMs and folks who are early days building things. I always joke that

building things. I always joke that people I'm like if you spend a few years like building product at Duolingo, your product sense is going to be 10x better than if you work at somewhere else. Um,

and I I I think this ritual is a big part of it. And I think it's also just like we care a lot about quality. And so

like somewhere else you'd get away with like crappy copy on a landing page or on an onboarding screen. Like that does not fly at Duolingo. And so I think that just helps level everybody up. And so

everybody's product sense is just way better and their taste is very sharpened.

>> So I think I compare that with you know when when people think of Airbnb they obviously think of like a a very design ccentric user centric um kind of company

but it sounds like maybe the rituals there didn't seem as effective as the rituals at Dualingo. Um, so if someone is listening and they're like, I have a pretty good, you know, yeah,

>> design culture at my company, but I don't have this kind of ritual Nikki's describing, what would be almost like the way to bootstrap the V1 version of a ritual like this at a company that that wants to maybe start doing something

like this?

>> Yeah. I mean, I think just having a clear it's a meeting that's on people's calendars and we have them at Duolingo every day. So basically you could grab

every day. So basically you could grab time every single day and it's like all the senior leaders in the product organization including the CEO is in that meeting every single day >> like a 30 minute block on the calendar every day.

>> It depends on the day. So we have different formats. There's a mini

different formats. There's a mini version and there's a longer version and you book different time depending on what you're building. There's a whole thing. Somebody needs to write about

thing. Somebody needs to write about this. Surprised it's not out there.

this. Surprised it's not out there.

>> But what that does again is so so it's easy for you to find time. Like you're

never like for the most part you're not typically waiting to go into this ritual which I've seen that not go well at other companies where you're like I can't get a spot for two weeks >> and so your team is waiting for two whole weeks and like again if you think

about >> exactly so again when you hear when you people are like they cringe right that's why because they've been like well I couldn't ship anything for a month because I had to wait and so like ours

is not designed in that way. Um, but I think it's so basically it's a it's a frequent meeting and it has very clear guidelines of who who signs up the expectations of what they're bringing and the format of what they're bringing.

Um, and then who is in charge of giving feedback. If it's a free-for-all and

feedback. If it's a free-for-all and there's 30 people in a room and everybody's giving feedback, like that is obviously not going to go well. Um,

so having clear people whose job it is to give feedback. Um, and making sure that those people who are giving feedback are very calibrated with what good looks like. Um, and I think just starting there, and you can even start

in a casual way where it's like, we're going to try this for, you know, a couple weeks. We're going to see how it

couple weeks. We're going to see how it goes. We're going to iterate on it. Um,

goes. We're going to iterate on it. Um,

and just kind of committing to make it better because again, you're probably your first version is not going to be right to start.

>> What What's the difference between this and what I might consider traditionally to be like a design crit, just like sitting in on a design crit? Is there is there a difference or is it is it kind of similar?

>> It's similar except this is all more functions come. So, it's not just

functions come. So, it's not just design. I'm pretty sure design crits

design. I'm pretty sure design crits still happen inside of Dualingo. I don't

go that go to them personally because it's typically just design and so like they're they're in Eliza problem when that's your only mechanism to give feedback, right? Is

that you're missing people in that room and so this is more across collab like typically the people presenting are like the team lead. Well, it depends. It's

like the people building the thing. And

so it's usually like a PM and a designer for the most part. Sometimes an engineer is there too depending on what it is.

But that helps again just making sure that you have multiple different disciplines and it's like the whole team. It's more of like a team focused

team. It's more of like a team focused ritual.

>> Got it. And the kind of thing that they might bring that Sorry Mark, but I'm just like super curious about this. It's

like what makes for a good like topic to, you know, if you show up today or tomorrow, well, I guess tomorrow's Saturday, but one one of these reviews and you're like, "Wow, that was such a good that was such a good one." Like

what what makes for a good one? what I

mean it's so it's literally everything like in order to ship something you're going to bring it through there so it totally depends but for example so I'm my team is working on math Dolingo Dualingo math and we're working on a new

lesson type that is introducing math to people for the first time and it's much more exploratory so there's sort of like a fun element where you can kind of play and interpret the math and so we needed to figure out like that's a hard problem

to solve right if you think about math there's thousands of math topics we had to build a format that's going to scale across all those topics. So if you are being introduced to quadratics or if you

are being introduced to slope or you know name it name a math concept linear algebra um you there's some way where that lesson type scales so you can actually interact with the math in a compelling way.

>> And so you can imagine that's like a that's a hard problem for anyone to solve any team to solve. And so that's something you would bring through at a couple different stages to be like here's how we're thinking about this.

Are we aligned that we want to do this and that this is a problem we want to solve? Thumbs up. Okay move forward. you

solve? Thumbs up. Okay move forward. you

go back again, you're like, "Okay, here's what we're going to do. Here's

our recommendation on what we actually want to build, get a thumbs up." Um,

sometimes we'll bring things in in a prototype format. Um, that was an

prototype format. Um, that was an example where we brought a prototype so everyone could try it. Um, which helps because then it's actually you can, you know, you can interact with something that is designed to be interactive. It

helps a lot. And so that you get really great, incredible feedback. Uh, and then you can implement that and it goes out into the app. So, we're working on that now. It's not out yet, but soon

now. It's not out yet, but soon hopefully.

>> Very cool.

And is it so is it up to the team to like decide what kind of feedback they're looking for?

>> You're just like, "Hey, I'm in the design process. Here's what I have

design process. Here's what I have >> like and how much?" Yeah. And how

important is that, right? To like be good about like what what you're asking feedback for.

>> I think it's always good to know like what you want feedback on because then you can help guide the conversation.

Typically a like somebody who's good at building product will be like we kind of we think this part is the tricky bit.

Like do we like our approach to this?

Like are we aligned? Um, and so I think that helps a ton. Um, but also just generally like the format of the meeting is designed to just give feedback on it

anyway. And so that I would say like

anyway. And so that I would say like like you're going to get all kinds of different feedback from different people. And so you can guide it as much

people. And so you can guide it as much as you can, but you're just going to hear things about the experience regardless if you like it or not.

>> Yeah. And it I have another kind of question here is like how do you how important is closing the loop with that group and and how does that happen?

Because I feel like that's like one of like good product cultures, too. They're

they're they really they're really good at like not forcing people to close the loop, but at least like just like bringing it back, but hey, here's what we did based on this meeting and here's what we learned and like basically like

you have that collective context almost and like so how does that work for Dualingo?

>> Yeah. So, we we will say in that meeting, we'll say you have to bring this back or you're good, go for it. And

then we have a Slack channel for little things because sometimes there's little follow-ups that like doesn't make sense to bring it back in that room. And so

like that those are those are looked at too, but for the most part like you'll leave the meeting and you'll either know like this is approved or please come back with these changes. And again like the rituals like it's very positive. It

sounds like it's it would be very pedantic and like like you know overly harsh. It's really not. It's meant to be

harsh. It's really not. It's meant to be like we are all on the same team. We're

trying to build cool awesome stuff and let's just make sure that we all understand the bar and then we'll just keep bringing things back until we think they're ready. H I was going to ask I

they're ready. H I was going to ask I was going to ask what you what have you learned about just from like being part of this culture and observing this ritual. What

have you learned about what makes for the most help helpful kind of feedback or like the like if you're giving someone advice on how to deliver better feedback as a leader to to the team that's presenting, what would you tell

them? Yeah, I mean I think just being

them? Yeah, I mean I think just being really clear and being able to articulate like what type of feedback you have really quick like quick is we've designed this to be a pretty quick ritual. So you don't you can't just like

ritual. So you don't you can't just like blab on and give a ton of feedback. Also

just like the most important things I think giving feedback on the things that really are the blockers to shipping is really important because obviously there's like endless improvements you can always make to things. Um, so it's like finding the right like what's the

bar and making sure that the V1 is just on the right side of the on the just past the bar, not too far past it, right? Um, and so I think that's really

right? Um, and so I think that's really important to make sure that you're giving really good feedback to teams just so they understand sort of like what they need to do next.

>> Yeah, I think for for me like when I've seen like the best product reviews is like when there's like a shared language of for feedback and like how to categorize feedback. Uh so for example

categorize feedback. Uh so for example like Arasana uh basically like product leaders that were giving feedback would have to like categorize feedback into like >> do which is like and then you have to do

this like just it's not even like a thing consider okay like I think there's something here like you know my product intuition tells me like you should really just think about this >> and then the other one it's like like

it's like a hunch try this right like and basically like depending on on those things you would have to come back or not and that was really helpful because then it's like because before before we had this shared language you'd have like the CEO say something you're like oh

like does that mean that we have to do that now like it was just like kind of like there's too much ambiguity but like I think once you get to like that that yeah like position is really helpful >> or it's like is it blocking feedback or not blocking feedback right and again

because again like there's an endless quality level you could be trying to attain and you don't want to do that because then you're going to waste everybody's time and you don't even know if the thing works yet because you want to get it out and so it's like again

there's like there's a bar and it's just like you want to be just past the bar to make sure that you're not you know wasting people's time before you know if the thing that you're even building is going to do what you think it's going to do.

>> Yeah.

>> You mentioned you said sorry. Go ahead

Mark. Yeah. Yeah. Go ahead.

>> Yeah. Just you mentioned that this ritual is like really helpful to like almost elevate the collective taste of Duolingo.

>> Yeah.

>> Is that because a lot of people are shadowing the meeting or or is it because like one person will go in and kind of share what happened like like how do how is that helping kind of like bring it to maybe like a junior engineer

that's like not the team lead but like who's working on that thing?

If you're enjoying this conversation, please check out the links in the show notes to support the podcast. Mark and I do this out of love, but to keep it going, we also need your support.

Thanks. And now back to the episode.

>> Totally. So, it is an open meeting and so basically anyone in the company can join and we tell people even if they're not in product or design or engineering to join it because it helps you calibrate on what good looks like. And I

also think like even if you're the quality bar should be everywhere inside of your organization. Like if you are writing a if you're writing a help article that's going to live on your help site like thinking about quality

there is as much as like the homepage or the landing experience that every single user will go through. um it's the same and I think a lot of times companies will like really they'll they'll be

cheap with their time on certain places and they'll cut corners and you can feel it when you use the product experience right is like people see those things and they notice it um and so I think having a ritual like that is just a

helpful way to scale taste across organizations um and then we always tell people especially when people are newer they're ramping like we have them spend a lot of time in PR because you'll learn just like the words that we use at

Dualingo and just like you'll learn about leader and like what they care about and you'll also get a glimpse of like all the cool products and get ideas actually from other teams and what

they're doing um at Dolingo. So you have to be careful that you don't spend all your days inside of our product ritual for the most part um which if you're so we have people who are reviewing them every day and it's definitely like a big

chunk of their job is just like being in those meetings and like being on the hook to review a ton of work.

>> Yeah. I'm curious. You mentioned that this meeting is a long-standing meeting that predates your tenure at the company, right? So you've seen I'm sure

company, right? So you've seen I'm sure lots of iterations of the meeting or how it's evolved over time. Has I mean with you mentioned nowadays the artifacts that get presented might be prototypes.

That's probably one way that AI is kind of influencing or shifting the way that this meeting runs. But have there been any other kind of like material evolutions to the way that the the the

way that this meeting is conducted that have been influenced by AI?

>> Yeah, I mean I think AI definitely changes it because the you're bringing things you're bringing different kinds of products through that meeting too. So

for example um we have a generative AI product that's called video call right and in video call uh you are talking to one of our characters and you're practicing speaking your language. Well,

you can imagine like static mock for that doesn't make sense anymore. And so

like you need a build and then you need to give you need to spend as a reviewer or somebody giving feedback, you need to spend a bunch of time trying to find the edges of quality with that generative prototype. And so like that is

prototype. And so like that is definitely a change. Um in math we're using AI to generate some math math content to make sure that we can like scale across all of the million thousands of math topics, right? we all

of a sudden are like unlocking giant chunks of content at one time. And so

you'll also bring a build through that room and then you'll have to figure out and make sure people have a time even outside of that room to spend with the content because it's a ton all at once.

Um and so it's definitely like pushing on I think the boundaries of what we can do in like a 30 minute meeting every day. Um but I think what you know we're

day. Um but I think what you know we're pretty good at iterating on our rituals and so I imagine that will continue to evolve over time. Um, one of the things that I think is so important with AI that I feel really strongly about and

you can tell when people do not do this other other especially consumer products is you literally if you're going to especially if it's any kind of generative AI, you need to spend a long

time dog fooding and like a really short dog fooding cycle with your with your team on something that maybe could have many different edges and be used in many

different ways like is not enough. And I

think like QA as a concept has really broken with AI and I I feel it when I use products where I'm like, "Oh my god, like clearly nobody looked at this."

Like immediately I can break it or actually I feel like that's like a this is going to be my new part-time job.

Like send me your app and I will break it. Like I will do it. [laughter]

it. Like I will do it. [laughter]

I will find the edges and I will get it to do things you don't want it to do.

Trust me. Um and I think like so that's important. So we do that a lot inside of

important. So we do that a lot inside of Duolingo too. We do that on the math

Duolingo too. We do that on the math team is we have dog fooding sessions.

We'll get together as a group. We also

just have things in dog fooding builds.

We'll put things in our path that are marked like dog food this so people when they're using they're doing their daily lessons will be doing it you know in these specific places and it helps a ton because it helps everybody see it. Lots

of eyeballs, lots of different types of phones. People trying to break it really

phones. People trying to break it really really helps. And you again like you can

really helps. And you again like you can always tell I can I can tell when a consumer product goes to market and I'm like clearly you were not >> you were not spending enough time thinking about that.

>> Yeah.

>> I'm I'm curious. I mean that's such a good point and I definitely have used those products myself as well and >> but it's such a hard problem because I feel like as you said like similar to you know going to these meetings and

getting feedback there's always ways to get better and I'm curious like >> do you have any principles that you've developed around dog footing when you know it's good enough >> because I think like [clears throat] you know you can spend time you can do you

can do two dog footing sessions but still like also the models are changing so like yeah how do you think about like when it's you know good enough. Yeah, I

mean I it's that that's a hard one because I think it also depends on the product that you're building and also not everything is fully generative, right? Like there's ways to use AI to

right? Like there's ways to use AI to help you create things that will be static once they live inside of your app. And like that's actually like

app. And like that's actually like that's much easier to dog food those types of things because you're generating things once everyone can look at it or even you can rely on user feedback if that's sort of your feedback mechanism and then you know if things

are good or bad and then you can change it one time versus generative experiences where those are harder um where people can kind of use it many different ways and even just like having your 10erson team try to dog food it

like it's not enough and so I think like there's there's a fine line there to kind of strike the right balance honestly but I think generally that's also where taste comes comes into play.

I think like one of the most important things that you can do especially in the AI times as a builder is like start to develop a really strong opinion of what good looks like and then if you're a leader inside of an organization like

help your organization develop an opinion on what good looks like and then make sure everybody understands what that looks like. Um because I think you know it's really easy to sort of like

push on the boundaries with new AI experiences and if your team is not aligned you're going to feel that in your product experience or you're going to ship something that's going to potentially be really punishing for your brand if you're not careful.

>> Yeah.

>> Go ahead. Go ahead.

>> You go. No, you go Ben. No, I was going to say because like I I was talking to a founder recently who is building something that is going to have a a it's like 10 people using their product might

have 10 different kind of user permutations of the user interface because they're using AI to dynamically evolve the experience in a in a way

that's personalized. And so they were

that's personalized. And so they were talking to me about how it's it's challenging to find people that can thrive in a design role for example or like a product person that can thrive

because the definition of good in a deterministic world you can kind of bring static mocks to the table.

Everyone agrees this is going to crush it. It's going to be great. But in a

it. It's going to be great. But in a world where like the pieces the UI elements and the UI components themselves are evolving in almost like a hard to predict deterministic non-deterministic way. I'm curious if

non-deterministic way. I'm curious if you've learned anything about how to kind of like bring taste into like is there a different way to think about taste in that kind of non-deterministic world.

>> I think it's just what you're doing is you're converting your taste into evaluators and so you need to you need to sit down and write it out and be like and you can have your team help. This is

a great brainstorm or you can have in your room and you're like okay again it's like aligning on what do we think quality looks like? What what do we think this thing needs to do? What

shouldn't it do? What does bad look like? What does good look like? And then

like? What does good look like? And then

it's breaking those into like a series of evaluators that we'll be looking at and guiding your experience to make sure that like it is helping deliver the or say the right things. Honestly, it's

passing through many filters that are trying to filter for many different things. And I think even if you do that

things. And I think even if you do that well, it's still because it's not deterministic like still going to do weird stuff sometimes or hallucinate or whatever. I have this theory that like

whatever. I have this theory that like the general like the culture or or the people are going to get one they're going to get way better at like understanding when basically just like experiencing non-deterministic

experiences and and giving them more flex and understanding that it's AI but we are we are not in that world yet like people will interact with something and they'll be like that was bad because they don't they have no idea how it was

built or what it's doing and I think that's a really risky place for people building products because you when your end user doesn't understand like they just suddenly will completely write your product off if you're not careful. Um,

and so I think it's like going through the motions again of like really thinking about what are we building? How

do we actually make it simple? So maybe

if it's if it is non-deterministic, we're doing it in a very constrained way that like still is like sandboxed so it's not going to do anything crazy or like hurt our brand. And I think that is something I don't I don't see a lot of

people really doing that today or doing it well. Maybe this is just a muscle

it well. Maybe this is just a muscle that we all need to build collectively.

the specific thing that the muscle that you're referring to is the ability to balance very opinionated constraint definition while doing what

>> still having some open-endedness, right?

Because again, if like say you're building like you know you're personifying a character in a way that's going to help you do something or whatever it you know it still it does need to be creative and and it needs to

feel like you need to feel like it is personalized to like what you're talking about. So like you can't constrain it

about. So like you can't constrain it too much otherwise the whole thing doesn't work. Um but ideally you are

doesn't work. Um but ideally you are constraining it enough where it's going to kind of like still stay on the path.

There is a happy path with these things.

And so it's like finding the right line to walk with it and then making sure that you're like building it in a way that does that. And I do think like sometimes, you know, just fully keeping

things open-ended like that's also, you know, like chat GBT wants to do that, right? Because they have so many users

right? Because they have so many users that are needing to do different things.

And so you can imagine like the way they're going to build something is going to be very different than if you are like I was trying a product called Tlons the other day where it's for kids these really cool characters that are like personified for kids. And you can

imagine like their evaluators are going to be very different than someone like ChatgBT because the use cases are so different. the product that they're

different. the product that they're building is so different. And so just like really thinking about it that way and figuring out like what what are we trying to build? How do we bring this to life in a way where it's going to be

safe and it's going to like help the user accomplish the thing that they need to do? Otherwise, you're just, you know,

to do? Otherwise, you're just, you know, you're risking just making your user mad right away and then they're not going to use your product, which is not good for anyone.

>> Yeah. And I think the implicit point that you made too is that I think there's a lot of companies out there that actually do really impressive kind of like they have really impressive products that are generative, but because they don't pay attention to that

last smile and maybe like the text like looks like there's some hints that you know this is very AI generated even though the output is actually really good. It makes the user underwrite like

good. It makes the user underwrite like all this amazing work that they've done.

Um, and then trust just kind of plummets. Be like, "Oh, I don't want to

plummets. Be like, "Oh, I don't want to use this product because they clearly don't care and clearly like >> it's a slop."

>> Exactly.

>> Exactly. Yeah. And the slop thing is real. Like it's, you know, it's hard to

real. Like it's, you know, it's hard to create really good product experiences that are not sloppy and like you need to, your team needs to be like laser focused on making that happen. And I

think things still slip through. And

again because this like larger consumer base has not been taught or doesn't understand what's happening with their tech that they're using like they are very punishing as they should be

honestly like um and I imagine like you know especially as like younger people are becoming more AI native and are learning this when they're younger and using it even in schools now like in 10 years that might not be a problem

anymore. I think like people will

anymore. I think like people will probably be a little bit more fluent.

Um, but today you're in this like risky place where you have to be really careful because people don't understand what's happening. Um, another thing that

what's happening. Um, another thing that drives me a little crazy is when people just like slap AI onto their marketing and they're like AI blah blah blah and like your end user doesn't care. They

just want your product. They don't they just want your product to do the thing that you're telling them to do or that you're saying that it's going to do. and

you putting a on AI on it for no reason.

Like maybe even as hurting what you're trying to do. Um you know, especially like in specific use cases like that does not make sense. Not everybody's

techy. Not everybody is going to care about AI or actually some people actively don't like it. And so you have to be really thoughtful about like how you're talking about what you're building too because that doesn't help in all cases for you to like slap that

into your marketing. And it is everywhere now. Like I was in um I went

everywhere now. Like I was in um I went through LaGuardia airport last week and there was like six ads like when I was on my little like I was on the walking that was like AI for the

>> crazy I'm like nobody cares nobody cares and so if you're marketer if you're a product marketer >> tread carefully [laughter] >> but it's yeah it's funny like what you just described

>> go come to mind is like the Salesforce like cloud thing like where it's like internet sl like or uh Yeah. Server. I

forget what their their anti-positioning was, but if someone can come out with a AI powered an AI powered product that like literally is like taking on an anti- AI positioning play, that's that's probably one way to stick out in the

>> Totally. I think that's very true. And

>> Totally. I think that's very true. And

again, I think that there's that sentiment because people are shipping really sloppy experiences and they're not using AI responsibly. And so,

consumers are mad about it and I think they deserve to be mad. And so I think that that's just going to push people who are building like they're just going to make people build better and build more intuitive, thoughtful product

experiences, but we're in that like weird gray zone time now where it's just like we're not there yet. And so that's interesting in like the next 10 years because I think builders will get smarter

>> and then consumers will a little just understand what's happening with AI a little bit more. Um, and ideally there's some nice moment where all that comes together someday, but we're not there yet.

And I'm sure those companies that maybe are have like a lower bar and what you're describing is also happening like at a meta level in the way they build product like for example they're allowing some of that slop let's say

when people are writing PRDs or you know when they're creating prototypes right like >> like in surface it looks good and sexy but then you look at the copy I was like hey this copy is trash like you know that's part of the product so I think

like >> everything kind of permeates um and how you do the small things probably translating to how you do the big things and and the end result which is the product. Um

product. Um >> yeah and that's again where like we keep coming back to taste because I do think one of the things that is really important in any kind of can you know this can be a startup even even when you are you know working on your podcast

like when you're writing your show notes like that is a moment where you are going to you know write things in a way to make sure that they are quality that it makes sense like all of those things

really matter and I think if you are leading an organization again it's like showing your team and helping them build their own taste so when they are doing little things or big things like they're all done in a way that's like aligned

with sort of what you're trying to do.

And I think a lot of people are sloppy is they'll they'll use the AI to to like write the landing page copy and then you read it and you like are actually thinking critically about it and you're like that doesn't make any sense or that

doesn't sound good. Like a person wouldn't write it that way. And so just being able to catch that stuff is going to be important especially if you're a leader because especially in latestage capitalism, right? Everyone's busy.

capitalism, right? Everyone's busy.

We're all shipping a ton of stuff. It's

really easy to use AI. It's easy to like lean back on that and be like, I'll have AI do it, but that's not always the right thing to do because it doesn't always ensure that you're going to ship something that's really high quality.

>> Where are you landing as far as like things people can do to hone or or improve their taste? Like, so we talked about this ritual internally, but is there anything else that you would tell someone who's like, I want to get better

at at having better taste? Like anything

that you find yourself repeating to people?

Um, so I always joke that this is this is my future Maven course because I think that this is something that there's not a lot of good resources out there that are doing this today because it's hard. It's also highly

it's hard. It's also highly contextualized depending on what you're working on, right? Like you can imagine the taste of shipping something like say you work on Slack for example like you're going to have a very different set of product principles than working

on a consumer product like Dualingo. And

I think that like one way to train yourself is just to use a lot of products and to think critically and ask yourself what is good about this, what is not good about this. I do this a lot.

I actually do this with my kids. So you

can start if you have kids. Uh I think you can start making your kids think critically when you're just out in the world and be like they'll be like, "I like this song. Why do you like that song? What's good about that song? Tell

song? What's good about that song? Tell

me more." Um or you take them to a movie and you say like, "What did you like about that?" "Oh yeah, what was your

about that?" "Oh yeah, what was your favorite part? Why was it your favorite

favorite part? Why was it your favorite part?" It's the same thing with building

part?" It's the same thing with building products. like it's not that hard. Um

products. like it's not that hard. Um

especially consumer products literally it's like how do you build the most simple intuitive thing and really looking at an experience like you could download any app in the app store and do your own product critique on it and be

like okay this is for elders who have need help with their passwords. How

might an how might somebody who is older navigate using this experience? What

would be hard? What's easy? You know

what is this design doing? Is it

helping? Is it hurting? like just kind of like thinking critically about things like honestly goes a really long way.

Um, one simple thing for taste building I think is just using a bunch of different products. And a lot of product

different products. And a lot of product builders do this, they do this already because they're kind of techy and so they're just trying things. Like that is just a great way to see how different people are doing different things and it makes a huge difference. Um, and it's

just a really effective tool. So start

there. Um, and I think that really helps. And then just listening to all

helps. And then just listening to all the good podcasts and all of the content about what good looks like will just help you kind of sharpen your own taste over time.

Yeah, I mean I feel like part of it is what you said right of like that like that discipline and exercise of just always try to ask you know why why do I like that what I don't like why why

I don't like that the other half though like when I think about like you know the persona of like my favorite designer that I'm like they have really good taste like everything that they do is just beautiful >> it almost sounds like they surround

themselves in like a beautiful world I go to their houses and like it's everything's like super clean and and beautiful.

So, and I kind of get the same vibe with you right now. I'm like I look at your back and I'm like man like you know like so >> what do you have to say there about like yeah like [laughter]

>> no about like you know what is your advice for for for like having people immerse themselves in like either physically or virtually in like a beautiful aesthetic world that can also

help you bring your standards up.

>> Yeah. I mean, I think it's I really think taste is just critical thinking at the end of the day. And it's like training your own critical thinking.

Like the same way you build an app experience is the same way you could design a room if you want to think about it that way. What do I care about in this room? What does it need to do?

this room? What does it need to do?

What's its function? Like how might I bring this to life in a way that like helps me achieve whatever that goal is?

And then just really thinking about how you do that with every decision that choice that you make. And I don't I think there are probably people who have incredible taste that maybe don't have

an incredible background, right? Because

they don't care. Like it's not I would say it doesn't cascade everywhere. But I

think like again it's just sort of like really choosing yourself to think critically when you're like creating an experience. And you can do this in so

experience. And you can do this in so many different places too. It's like if you're going to host Thanksgiving, ask yourself like actually set an intention.

How do you want people to feel when they're at your Thanksgiving experience?

Like how do you bring that to life? What

do you serve? You know what I mean? How

much wine do you buy? Things like that.

>> Yeah. How how full do you want people to feel when they leave? Right. Like it's

like these kind of questions. And I, you know, I agree. I think a lot of people like it's kind of easy to say what you hate about a product or or that that this sucks or this is frustrating or

it's so slow, blah blah blah. It's easy

to complain. I think it's a little less easy to pinpoint specifically what bothers you about it. And that's a that's a good skill to develop is to to say this is too slow or when I click

this it's not fast enough or it takes me somewhere that that I'm confused about or whatever.

>> But I think as you to me like a a real leveling up is to be able to pinpoint the specific reasons something feels good >> and like you mentioned like what do you love about that song? But I but even

though it's great advice I think a lot of people don't develop that. It's

almost like we're we're wired to fixate on the things that are not working versus the things that are working.

>> Yeah. Yeah.

>> So, I think it's a little bit of a harder muscle like when I find myself just not complaining. I'm trying to nowadays like catch myself in that moment and being like, why is this work?

Why do why is nothing bothering me right now? What's working about this,

now? What's working about this, >> right? [laughter]

>> right? [laughter] >> It's kind of a way of trying to be mindful, I guess. But

>> yeah, it's just being pre like I guess present in your environment. The other

thing that I think really helps is, and this is something that definitely like you are for the most part like a builder to have access to something like this, but if you're going to put out anything in the world, do user research. Watch 50

people use your product experience and that's another way to build a taste because you're like, "Oh crap." And

you'll be surprised. Like even senior people like will not catch a thing. And

then you send it through a bunch of like 50 kids see it or something and you're like crap. Like that didn't work at all.

like crap. Like that didn't work at all.

Um, and I think like when you do that a hundred times, you get really good at reading like how things are going to kind of like live with different audiences and different groups. That's

one of the things that I think has been so fun working at Dualingo. Um, when I joined, so I've been there for like four and a half years. And when I joined four and a half years ago, I I didn't join because I was like, I I'm going to level

up my product sense. I was just like, really fun consumer app. People seem

cool. Let's do it. And what I realized is like Dualingo is in so many million.

It's in so many pockets of people all around the world. Like I see people using Dualingo when I'm out and about regularly and watching different people use the product experience has really

sharpened my product taste, my product sense, my ability to create good experiences because there's just so many people that it need that needs to work for. Like my grandma uses Dualingo and

for. Like my grandma uses Dualingo and my six-year-old daughter does and it needs to work for both of them. And so

like I think Yeah. And so I think that's where you know even with like if we think about AI like with consumers like you need to build things that are going to work for you know not every product

is overly tailored on like it's it's not going to be like either hyper tailored or work for everyone. Um but just again being really careful about when you're building experiences really making sure the person that you're building for is

like going to be able to use it has the right context or knowledge to understand the nuance around it. And that's I think that's where AI get continues to get itself into trouble or I see things with

just like you know kids using it and then they they happen and it's like well yeah if you if you think a kid is going to use your product like you should make sure that you have those edge cases like figured

out and I think just like not enough of that is happening right now and so I'm hoping again like I think it will it will happen in the next five years or so. I think it's still so early but

so. I think it's still so early but there's a lot of work to be done to get there.

>> Yeah. I mean I think taste is like like a big like I think taste kind of has made like a big comeback. I mean it's always been like super critical for product but like with AI I feel like it's like oh like now it's like what

differentiates like the normal people to 10x like product builders. I feel the other one that's like big which in a way is like a different flavor of taste is

like kind of strategy or judgment and and for example like I was thinking like wow for Dolingo like you could do so many like with like now with AI you could do so many things right like you

could you could literally just like someone could sign up into the to your app and you could literally ask them hey like what do you want to learn and you could generate like different lessons and you could ask them hey like how do

you prefer to to learn do you like to be like tutor do you like lecture plus then questions. You just like to read stuff.

questions. You just like to read stuff.

So yeah, like how I don't know, it just sounds like a fascinating time to to be building product and and to have a strategy. So So yeah, I don't know if

strategy. So So yeah, I don't know if like if the way you think about product strategy has changed at all with like the, you know, with this new technology.

>> Yeah. I mean, I think because you have new tools now, it's just like your end product is just going to change based on the tools that you're using. And so I think it'll be really interesting to see like I think this is just for all consu especially consumer experiences. I think

it's going to be really fascinating again to be to see where we're at in like 10 years and to see like how things have evolved as like as true consumers, you know, everybody like crossing the chasm if you will, right? As more and

more people adopt and like become more familiar with new ways of building or using those products. Like I think it's just going to be really interesting to see how it changes like how you're building things and what your product

strategy looks like. But I think the TLDDR is that it has definitely changed the product strategy. Like the way especially in in Dualingo math specifically, AI has been so helpful because it was just going to take us a

really really long time to create math content for 10. It's thousand

technically thousands of topics for math. Um, and so now that we can do that

math. Um, and so now that we can do that and we've proven that we can do that, you can imagine that drastically changes our product strategy. And so I think like this is going to be really helpful to lots of different types of experiences because it just like opened

doors that were maybe closed before because you just never would have been able to build it. And so I think it's still like early days and we're still figuring out, you know, both at Dualingo and I'm sure in many other companies like how do you how do you change how

you're working and how do you also just like tweak the product experience in different ways. But I definitely think

different ways. But I definitely think like you're probably doing it wrong today if you have not iterated on your process based on all of the advancements that are happening in AI in like the last especially like the last year and a

half. It's like a totally different

half. It's like a totally different landscape which is also fun. Um I read this book recently about um how to basically it's about motivation motivating people and one of the things

that demotivates people and I think this happens like later stage career for a lot of people because you've been doing the job for like like I've been in product roles now for like over 15 years which wow it's like makes me feel very

old. Didn't say that out loud. Um but

old. Didn't say that out loud. Um but

one of the things that is a pillar of motivation is your mastery of like what you're working on. And one of the things, and this is where I think there's been such an explosion of AI content on LinkedIn, is that it's like

fun for people to build again because they were sick of doing it the way that they were doing it before. Um, I'm

surprised nobody has written about this yet, but like it's so different and it's changed how you're working. So, everyone

has to relearn and try things a different way. And I think that's just

different way. And I think that's just very energizing for a lot of people because they actually get to like try something new, do it in a different way.

>> Such a good point. It's such a good point that like because I guess what Chachi PT came out 20 >> 2024 three

>> yeah 2023 you should know >> yeah I think like the end towards the end of 2023 like okay it's like H2 2023 I think that's feels right >> that's that's right that's right because I I remember yeah I started

>> November 30th 2022 just lift it up >> okay 2022 wow okay so we're 30 three years we're we're in we're we're coming up into year four I guess all this stuff. But I I think the point you're making is so good because I do

remember >> call it end of 2021, early 2022.

Um people forget the vibe in tech and it was like not just market like the IPO markets shutting down, Zer coming to an end, but also on a personal >> Yeah. Everybody's getting like laid off

>> Yeah. Everybody's getting like laid off that was like post Yeah, >> totally. But but to your point from even

>> totally. But but to your point from even even if those were not enough to make it a bad time, everyone was basically in the hyper mature phase of SAS and like

of traditional software development.

>> And and to your point like now >> this is it's a new boom wave. It's a new >> there's almost like this reinfusion of energy and and

vigor that's making people excited again to work and build in tech. And I think everyone's feeling a simultaneous, at least everyone I talked to is feeling a simultaneous, oh my god, this is the

most exciting e time ever and simultaneously feeling it's I can't keep up with all of it. It's exhausting. and

how am I supposed to like do my So I'm I'm curious like if you've developed any um I don't know like any heruristics for

almost like when you're in too deep on something or you're you're like need to pull back and because you're I you're obviously a very naturally curious person and this is kind of [clears throat] a challenging

time to be a naturally curious person because it's all interesting. So like h how are you determining maybe like any heristics on where to hold yourself back? I feel like I'm not doing this

back? I feel like I'm not doing this well to be honest. Like I think I definitely fell down a rabbit hole and I'm interested and like needed to read all the books about all the things. Like

that's like my sort of toxic personality trait which makes me just like I'm just like a curious person. So I I think again like I just like I need to know all the things and read all the things.

So I should probably be touching grass more to be honest. You guys figure that out. Tell me. I'm the last person to be

out. Tell me. I'm the last person to be giving advice on that. I will say I have I'm a big believer in just making sure you have hobbies and you give make time for those hobbies. That sounds dumb.

Really, it does. I personally love to read. I read tons of fiction. I read at

read. I read tons of fiction. I read at least a book a week. Like I'm one of the most well read person people you will know. Um and I make sure that I make

know. Um and I make sure that I make time for that. And it's not all like eight books about AI, although I read those two occasionally. I'm reading I'm actually on the book five of the Thursday Murder Club series right now,

which is about some geriatric um crimeolvers in the UK. and it's really cute. So, if anyone needs a very

cute. So, if anyone needs a very delightful read, they literally are they're in like in a nursing home and they solve murders. It's very funny. You

should read it. But yeah, I think like making sure that you're not spending all your time reading about AI like who especially as a builder. I think like some of the best builders that I know are people who are really well read on

like nonAI things like on fiction or they go to the museums all the time or they travel all the time. And so I think you have to remember that taste is more than just like you watching user research. It's also you in the world

research. It's also you in the world experiencing things, being a part of culture. And I think that's going to be

culture. And I think that's going to be more and more important. And I also think AI in some ways is going to push us to more of those experiences because, you know, you spend more and more time on your phone. And I think people are

going to crave more of the in-person interaction.

Um I think you're going to you'll probably see a wave of products that focus on that stuff again. I think that kind of happened in the like mid2010s.

There was a lot of that like come together, we're going to do things IRL and then it kind of fell away and I think it'll come back as like more and more people are spending more and more time on their phone which I'm I'm all for. I'm all for that.

for. I'm all for that.

>> Yeah. Yeah. I don't think it's necessarily bad to go really deep into a rabbit hole. Like I actually think if

rabbit hole. Like I actually think if you have the ability to do that like it can be a superpower, right? Because like

I think you like especially if you're curious >> but I do think like so I don't know if it's like you have to hold back from going down into a rabbit hole. I think

that what you have to do is like take, you know, take breaks and and get perspective so you can go and apply it into whatever field or situation you're in. Um, but yeah, I do think that super

in. Um, but yeah, I do think that super like being curious is such a superpower that I think a lot of us in product take for granted. But I I think you got to

for granted. But I I think you got to listen to that. I just probably more more good than bad there.

>> I think as long as you're taking >> Yeah, exactly. That's what I was going to say is like what's been most exciting to me about tinkering cuz I'm also I'm in the bucket of like growing list of things I want to play with and tinker

with and not enough time um even without a full-time job just enough enough things going on in life that what I [snorts] find helps me is to have is if

I can bring a lens of what I'm trying to do for example like having this podcast if I see something if something crosses my radar that's like oh I could this could maybe help us solve a problem for

the podcast. It adds like some escape

the podcast. It adds like some escape velocity to that that thing that gets it to the top of the list to tinker with.

>> And then things that just like don't seem to have any immediate applicability to anything I'm actually like already interested in doing seem to kind of fall down in that list. Seems to be how at

least how my my cataloging um stack ranking works. But I'm guessing everyone has some flavor of that going.

It's like the stuff that does seem relevant probably pops to the top of the list.

>> Yeah.

>> Um cool. Well, we could keep talking about this for a long time. I want to be mindful. I know we all have some stuff

mindful. I know we all have some stuff we got to get done before the weekend.

So, is there anything else that we haven't talked about that you think is top of mind or interesting that, you know, you want someone taking away from

the conversation when it comes to like how to how to just improve the way that they build right now, especially in the consumer side?

>> Yeah. Yeah, I mean I think it's just like again having your own sense of jud self judgment when you're building things and in thinking about your own taste in like how you invest in it because I think that's what's just going

to be important in the next you know x years as built for builders specifically. And I think like there

specifically. And I think like there there are so many ways to invest in just developing your own sense your product sense and your taste that is just going to make you a little bit better than other people that are also tinkering and

building. And so I think maybe that's my

building. And so I think maybe that's my takeaway is like that's not wasted time like you are that is that is only going to help you as you're kind of shipping things and so you know invest in

yourself and your own taste because it'll take you further.

>> Have you found any any because I think something that I think product people are were bad at is like just prioritizing these things that don't have like immediate um kind of like value or that you're going to get

something out of it or that moves a needle or an outcome. the things are like a little bit more novelous. So like

have you seen any good ways that maybe that people have used to hold each other accountable to develop taste together or or any exercises you're like huh maybe you should get a group of friends and do

this like yeah any any ideas there?

>> Yeah I mean I think like Ben was just saying this but it's like find the things that are like interest like borderline interesting to you. Like I

think if you're like I'm going to have to spend a bunch of time doing this and it's not things that aren't interesting or don't like fulfill me in some way like it's not going to happen. And so

like figuring out what those things look like maybe you do like using other products and you think it's really fun and you like to try things so you're you're carving out time to try things a couple times a week and just like taking notes or it's just reading like I think

I need to read more fiction or science fiction is really relevant right now because like we are living in the sci-fi times all of a sudden. Um and so maybe that's really fun for you and so that's how you're doing it. Um or it's maybe

just like you know taking some courses on design and Figma like even learning how to use Figma and forcing yourself to design UI is really impactful. Um we

just put our whole product team through Figma training at Duolingo because just helping you actually create your ideas is going to even help when we're you know you're using generative AI to

create prototypes and things like that.

And so like you could do that. So, I

would say like figure out what's interesting to you and like lean into the places where you can like work on your taste that is interest adjacent because I think it just means you're gonna actually stick to it. But

otherwise, future Maven course someday, maybe by Nikki. No promises, but >> let's we'll talk about it. Talk about

it.

>> Yeah, I would sign up for right now.

>> Okay. All right. I'm counting on it.

>> Uh and I would say if anyone's listening, you want a sci-fi recommendation just because it's come up, I would recommend Hyperion. Amazing

book. uh lots of AI stuff ahead of its time and just a great story. So if if someone's just looking for an entry point into sci-fi, go go check out Hyperion. Have you read it, Nikki?

Hyperion. Have you read it, Nikki?

>> I have not, but I will add it to my list.

>> It's amazing.

>> We'll add it to the show notes.

>> Add it to the show notes. Dan Simmons,

excellent author. So, okay. So, in

closing, if someone's listening and they want to learn more about what you're thinking about, if they resonated with some of the stuff we talked about, where can they go to follow you and learn more

from you or hear more from you? And then

two, how can they be helpful to you?

>> Yeah. Um, so you can follow me on LinkedIn. I would say I post most

LinkedIn. I would say I post most frequently there. And I'm uh my it's

frequently there. And I'm uh my it's like linkedin.com/nikscarststead.

like linkedin.com/nikscarststead.

And then I also have a Substack, which I am terrible at writing on, but I occasionally when I need to bust something out, I will. And it's

nikki.substack.com.

And then how can you help me? Oh, that's

a good one. Send me book recommendations always. Actually more. I feel like I

always. Actually more. I feel like I need more sci-fi in my life. I read a lot of just like actually I just feel like I read whatever is like interesting in my mind any given day. Um but also

any just like any substacks or really good novel content like that is really interesting that is kind of probing at the edges of this time we're living in is also really fun for me. And so if

anyone has any recommendations there I'd love to hear them.

>> Amazing.

>> I got you on the the sci-fi book recommendations and >> yeah I'll hook it up with you after this. Okay. All right. Well, that's a

this. Okay. All right. Well, that's a wrap. Thank you so much for joining us.

wrap. Thank you so much for joining us.

This was great.

>> You got it.

>> Thank you.

>> Thanks, y'all.

>> Bye, everyone.