UXRConf 2025

[Music]

[Music] Hey.

[Music]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

Yeah. Heat.

Yeah. Heat.

[Music]

[Music] Woo!

[Music] Heat. Heat.

Heat. Heat.

[Music]

Hey.

[Music]

[Music]

[Music]

Woo! Woo!

Woo! Woo!

[Music]

Heat. Heat. N.

Heat. Heat. N.

[Music]

[Music] [Music] [Music] Hey hey

[Music]

Hey hey hey.

[Music]

[Music] [Music] [Music] [Music] [Music]

Heat. Hey, Heat.

Heat. Hey, Heat.

[Music] Check check.

[Music]

Hey hey [Music] hey. Heat. Heat.

hey. Heat. Heat.

[Music] N. Heat. Heat. N.

N. Heat. Heat. N.

[Music] [Music] [Music] Yeah. Hey.

Yeah. Hey.

[Music] Hey. Heat. Heat.

Hey. Heat. Heat.

[Music]

[Music] [Music] [Music] Heat.

[Music] [Music] [Music]

Heat. Heat. Heat.

Heat. Heat. Heat.

[Music]

Hey, hey,

hey. Heat. Heat. N.

hey. Heat. Heat. N.

[Music]

Heat. Heat.

Heat. Heat.

N.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Hey hey hey.

[Music]

We are Heat. Heat. N.

are Heat. Heat. N.

[Music] [Music] Heat. Heat.

Heat. Heat.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey hey hey.

[Music]

Hey hey hey.

[Music]

[Music]

[Music] [Music] Let [Music]

me let

me I know you want Heat. Heat.

[Music]

[Music]

Hey hey hey.

[Music] [Music] [Music] Check. Check.

Check. Check.

[Music] [Music] [Music] [Music]

Everything. Heat. Heat.

Everything. Heat. Heat.

[Music]

[Music] [Music] [Music]

Black crack.

[Music] [Music]

[Music]

[Music]

[Music]

Hey hey hey.

[Music] Hey,

[Music] hey hey hey

[Music]

hey.

[Music]

Yeah, hey yeah.

[Music] Hey hey

[Music] hey.

[Music] Yeah,

oh yeah.

[Music]

Hey hey hey.

[Music]

[Music]

Hey. Heat. Heat.

Hey. Heat. Heat.

[Music] Heat. Heat. Heat.

Heat. Heat. Heat.

[Music] [Music]

N. Heat. Heat. N.

N. Heat. Heat. N.

[Music]

[ __ ] [ __ ] Good morning everybody and welcome to the last day of research week. I can tell it's the last day of

week. I can tell it's the last day of research week because we have a long ass line for coffee right now as people trickle back in.

And I know some of you were out late last night. I'm not going to lie. I can

last night. I'm not going to lie. I can

see it on your face. Anyway, um I'm really excited for today. Today we're

highlighting a bunch of amazing talks that we really think could apply to everybody. But I will let our host for

everybody. But I will let our host for the day introduce that. Um before we get into that, a quick thank you as is tradition. Oh, can we get the slides up

tradition. Oh, can we get the slides up at the in the back? Perfect. Uh big

round of applause for our wonderful sponsors.

Thank you again for all that you do to bring affordable highquality learning and connection to our community. We

really couldn't do it without you and we're so excited and happy to have you here. And for today for UXRC, we have a

here. And for today for UXRC, we have a title sponsor and their name is Lookback and the CEO of Lookback, Henrik, is here to say hi. Can you come up for a minute?

Thank you. Yeah. So, yeah, I am Henrik. It is

you. Yeah. So, yeah, I am Henrik. It is

true. And uh this is the eighth consecutive time we sponsor UXRC comp.

So always fun to come here and see this.

Thank you. Uh obviously uh it's been so great to come back every year. It's

better and it's going to be better next year too. But let's start today and

year too. But let's start today and enjoy today before we get excited about next year. But uh huge shout out to the

next year. But uh huge shout out to the team. Of course I'm pointing to you,

team. Of course I'm pointing to you, Alec, but Maggie is doing all the work and no there's a lot of people involved in this. Of course, thank you to everyone

this. Of course, thank you to everyone who makes this happen. Uh people are here tuning in from I'm looking at the camera. Everyone on YouTube. Uh it's

camera. Everyone on YouTube. Uh it's

great. Uh all the volunteers and of course all the speakers. So what's

really cool over the years is that the speakers like all the old school ones are coming back which means they liked it. Uh but they're not talking about the

it. Uh but they're not talking about the same thing. They're always bringing

same thing. They're always bringing something else which means they're interesting. And then there's new people

interesting. And then there's new people coming too. So that's like a business.

coming too. So that's like a business.

You want like new customers and you want the ones that came here to come back. So

you've built a great thing. Uh thank you so much everyone. Um so a few things about Lookback. We are a tool for

about Lookback. We are a tool for qualitative research. Um we've been

qualitative research. Um we've been around for 10 years. If you go to our website it will say that we are built for research powered by AI and grounded in humans. And I think that's true. So

in humans. And I think that's true. So

we're built for research. Over these 10 years of researching researchers doing research uh we picked up a few things and we built an experience that is not video conferencing. I know a lot of you

video conferencing. I know a lot of you because you come to me using some other, you know, uh, let's not name names, but some platforms that are not built for research to do your moderated stuff. If

you come to us, you're going to like it way more. It's built for you, whether

way more. It's built for you, whether you're the moderator, the participant, or the observers. Uh, you can do all your moderated and unmodderated stuff.

You can uh bring your uh own participants or you can uh use our new integration with user interviews, our best friends over there, also original

sponsors. Hello, user interviews. You're

sponsors. Hello, user interviews. You're

the best. Um, great panel. Um, so you can do all of that stuff and then surface all the insights. You can

involve your stakeholders early. You can

actually, I think we're the only platform where you can research your stakeholders before you research your participants, which is very important.

So, uh, bunch of cool stuff like that.

You can come see me and, uh, know more if you want. Um, powered by AI. Of

course, you have to be. Alex said you can't be at the conference if you if you're not. Uh but we are uh we are

you're not. Uh but we are uh we are enhancing we're doing some of that lowhanging fruit stuff. So we're

enhancing the stuff that the humans are already doing like note takingaking and finding things and stuff like that. So

uh check that out. Um and grounded in humans. Uh I love all of the stuff that

humans. Uh I love all of the stuff that people are doing. I'm looking for outset here. Like there's a lot of cool AI

here. Like there's a lot of cool AI stuff uh where you can automate things.

Uh but ultimately our stance is that um the more AI we get the more humans are going to be important. So whether that's like real humans understanding real humans doing real human stuff u or

whether it's like humans doing some research at least with humans and then they can expand into doing some AI moderated stuff too but that's what we do and the most important thing I want to say is that I have a lot of swag like

t-shirts and stuff and I know what I just said sounded like a pitch but I'm not here to sell subscriptions. It's way

more sinister than like than that. Uh

I'm going to pick your brains on things.

There's a sofa over there. Please come

sit, unbburden yourself, tell me about everything that is terrible in your job.

Uh you have a chance to win a product that might help you with that eventually when we figured out how to solve that problem. Um and uh worst case, you get a

problem. Um and uh worst case, you get a t-shirt. So come see me. Don't be shy.

t-shirt. So come see me. Don't be shy.

Um you don't have to give me your address or anything. Just like grab some swag. Uh thank you, Alec, and everyone.

swag. Uh thank you, Alec, and everyone.

Have a good night. Uh good evening. Good

day.

I told y'all people have been out drinking last night like what am I what are we talking about? Uh interesting. Uh very cool. So

about? Uh interesting. Uh very cool. So

um this will be my last time uh on stage here. So I have some very important

here. So I have some very important thank yous to do before I introduce our wonderful host for today. Um can we get a big round of applause for our volunteers real quick?

Thank you all. Y'all are the real MVPs running microphones, checking people in, and being allaround helpful. Thank you

so much. Um, I also want to put uh give a big shout out to um not only Maggie, who's here somewhere, but also Aaliyah and Eevee who are back home in Canada

who are monitoring the live stream, who are literally making live edits on decks while the presentations are going on to make sure everything looks crisp. So,

please a round of a a round of applause for the learners team if you had fun today.

Uh, awesome. And then I just want to leave you with one final note. Um, I

know it's been a challenging few years.

Anybody had a rough go at some point in the last few years? Raise your hands up.

Great. Yeah, a lot of us. Um, look, I've been talking to a lot of people over the last uh few years obviously um and especially recently. And I got to say

especially recently. And I got to say I'm I'm pretty sure we've bottommed out.

I think it's only up from here. I in

fact I am so certain that it's only up from here. I think we're going to have

from here. I think we're going to have to grow this event again next year so even more of us can come and connect and hang out. Um so I'm very excited about

hang out. Um so I'm very excited about that. But I got to say it's not going to

that. But I got to say it's not going to happen by accident. The whole point of this whole

accident. The whole point of this whole week is so that we can learn from others what's working, what's helping them accelerate their their career, what's helping them accelerate the quality of

their craft, the amount they're able to produce, the way that they're able to help their teams. And it's on each and every one of us to figure out how to get better and incorporate this stuff and then come back next year ready to teach

people what you've learned as you've experimented and tried new things yourselves. And it's also on each and

yourselves. And it's also on each and every one of us to find somebody else who we can lift up who's maybe a couple steps behind us. I know lots of you have made some wonderful connections today whether you're here in person, you're

watching online and it's so fun to get together as a community because a lot of the time we don't have these spaces to do that. But

I really want us to try our best to reach out and make a new friend, make a new connection, reach out to somebody who's a few steps behind you and say, "Do you want to grab a coffee? Do you

want to have a call? Let's talk about how things are going for you because I promise you everyone in this room. Every

one of you has something important to offer somebody else. So, if you can please join me in making a little commitment in our minds right now to lift somebody else up this year and then

we'll come back next year and have a a great ass time and have a big party again and and Yeah. How does that sound?

Is that good?

Amazing. Now, one final story as I introduce our wonderful host for today.

I have to tell you how I met this guy. I was in Toronto and there was a

guy. I was in Toronto and there was a random Twitter account that I saw that was interested in Arsenal and UX

research who lived in get this St.

Catherine's Ontario. What? But from

Kenya I think originally. Is that right?

Anyway, we get we get into the DMs as one does and we end up going for coffee at Jimmy's Coffee on Portland. And I had no idea what to expect. I was like, "This guy may be about to take me to a

rave. He may be about to take me to a

rave. He may be about to take me to a soccer match. We might get into some

soccer match. We might get into some shenanigans. I'm not sure." But that in

shenanigans. I'm not sure." But that in I think 2017 or 2018 was the very beginning of a long and beautiful friendship with someone who I have known to be one of the best storytellers I

have ever met. Uh this host for today is somebody who's led research at some incredible teams. He is somebody who has taught me a lot about research and I am so excited to welcome my friend and soon

to be your friend to host UXR comp. His

name is Roy. Roy, can you come up on stage and take it away? Please give it up for Roy. He is way too kind. Way too kind.

Roy. He is way too kind. Way too kind.

And I I think it was earlier. I think it was 2015ish.

Twitter handle. Okay.

Okay. Well, hey, welcome to UX Arcom 2025.

Um, I'm I'm skipping ahead too quickly now. Alec did say we've we've

quickly now. Alec did say we've we've known each other for a few years. And

before we get going with the day, um I just reflecting on the journey that UXR comp has been on, I would love to know if anyone was here for the original UXR

comp 2019 2018. Anyone was you can put your

2019 2018. Anyone was you can put your hand up if you were here. Just that guy.

Okay. If you are online and you were there, type a I was there just so folks can know.

It's been incredible to see this journey and like Alex said, I've known Alec and Maggie from before UXR comp was a thing when it was just UXR Toronto. And the

thing that was so marvelous to see was two people who put together the very best meetups in the world. Have you been to meetups where you show up and you're

just like, what what is going on here? Like this is a mess. some the

here? Like this is a mess. some the

speaker is just being told to go up five minutes before like you're gonna speak today, go up. That was never what UXR Toronto was about. They put so much care, so much craft, so much like grit

into making it happen. And that laid the foundation for what UXR Comp is today.

So before we get going, I know we've already done some shout outs. We've done

some applause, but I want to take a moment to appreciate Maggie and Alec for all they have done to bring us together.

This doesn't exist without them. So,

let's take a moment and give a huge round of applause to Maggie and Alec. I love you

Alec. I love you guys. All right, so I'm your host. My

guys. All right, so I'm your host. My

name is Roy Opata Oende. If you really want to get close to me, call me Opata.

But Roy is totally fine. Both are great.

I'm your host today, but maybe more importantly, I'm sort of your friendly conference

bully. Okay. Now, what does a friendly

bully. Okay. Now, what does a friendly conference bully try and get you to do?

What I'm going to try and get you to do, encourage you to do is today, I want you to take a step back. I know we all have our phones on

back. I know we all have our phones on us. Some of us have our laptops out.

us. Some of us have our laptops out.

Let's be honest people, there are no research emergencies. As my lovely

research emergencies. As my lovely former manager Jane used to tell me, it's going to be okay to take a step back today and allow yourselves to take

in the work the thoughtfulness of what the speakers are going to give you because something that they say today could make an impact in

your work and in your life. But it's not going to happen unless you take a step back. Don't grab the phone. Just let it

back. Don't grab the phone. Just let it sit in there awkwardly. Don't pull out the laptop.

awkwardly. Don't pull out the laptop.

Just let it rest and give yourself just today to take this in for the good of yourself, for the good of your team, for

the good of your life. You never know.

So that is my conference bully request.

I will not be singling out people, but I'm watching you.

It is time for our first talk of the day. Now, Brad is someone who um I just

day. Now, Brad is someone who um I just think is supremely intelligent. I've got

to have a bunch of conversations with them, and I'm really excited about the session where they're going to just talk about, hey, research is partially broken, but there's something that we

can do about it. So, let's give a warm UXRC welcome to Brad.

Thank you. Good morning everyone. Uh, welcome

you. Good morning everyone. Uh, welcome

to UXR Conf 2025. Um, there's something really important that I want to talk to you about today. So, my name is Brad. I

use them pronouns. Um, super excited.

Let's get started. As I said, there's something very, very important um, to me in my life that I want to talk to you about. Dance. That's right. We're

about. Dance. That's right. We're

talking about dance. So, you're probably wondering like, Brad, what what it it's UXR comp. Why are we talking about? So,

UXR comp. Why are we talking about? So,

outside of my research life, I'm also a modern dancer. I'm in a dance company.

modern dancer. I'm in a dance company.

That's me. Um, from this work called uh The Lovers by Anna Saklo. I'm in the Saklo Theater Dance Ensemble. Um, yeah.

So, why are we talking about dance? We

use this one word um to refer to a lot of different things, right? There's a

lot of different types of dance. They

all have their own like use cases, if you will, right? their own uh goals, their own engagement models, feelings, etc. Um, so let's talk about some of those types of dance. There's theater

dance, right? Ballet, Broadway, modern dance, etc. There's ballroom dance, more of a like social sort of collaborative effort. There's these like ritual

effort. There's these like ritual traditional indigenous dances that are maybe more about um sort of yeah, like honoring your ancestors. You know, I'm

certainly not an expert in this, but um there's all these other types of dance I could mention. There's K-pop and capoera

could mention. There's K-pop and capoera and yeah, just a bunch of different types of dance. There's whatever you do on the dance floor. Um we won't talk about that too much, but we call these

things dance. Um they're they're all

things dance. Um they're they're all very different. So, we can start to

very different. So, we can start to classify these different types of dance on a variety of spectra, right? So,

there's uh presentational versus experiential. Is the purpose to put on a

experiential. Is the purpose to put on a show or to have a good time? There's

professional and casual. How much

preparation went into this? There's

again ritualistic versus informal. Is

there a series of steps that you go through in order to partake in this action? And is the focus more collective

action? And is the focus more collective or more sort of individualistic? So, if

you're doing this like group dance, obviously that's more of a collective feel versus going out on the dance floor and doing your own stuff. There's

another word um that is really important to us that I want to talk to you about.

Take a guess. What do you think it is?

Research. That's

right. So, we use this one word to refer to a lot of different things. We say,

oh, we talk about qual versus quant and generative and evaluative, all these things, right? Um there's also this

things, right? Um there's also this whole problem of like what do we even call ourselves? I'm a user research, UX

ourselves? I'm a user research, UX research, design product. I just start I just research that's what I call what what is my team what do I do etc. Um so

I think yeah there's there's a problem here where we have this word and we're not really being specific about how we use it. There's another dimension that I

use it. There's another dimension that I think we're not talking about enough. So

we have um people are starting to there's some chatter about it. There's a

lot of talk about it this week at some of the other uh conference events. But I

want to talk to you about strategic versus tactical research. Uh, and

specifically how that differs from generative and evaluative because a lot of times we say we're doing generative work and we think we're being strategic, but I personally think they're orthogonal. And I say this because you

orthogonal. And I say this because you can do generative work, which is actually very very tactical. What

problems exist with signup? Like you

have a very specific thing you're trying to accomplish, but there's you're trying to discover what's out there. Discovery.

We talk about discovery research. On the

flip side, you can do evaluative work that's highly strategic. Which of these new product adjacencies should we invest in? So, I don't think we can classify

in? So, I don't think we can classify the work that we do and say, "Oh, it's generative, therefore it's strategic."

We're going to hammer this point home a little bit more um with this little ven diagram. So, the really the only place

diagram. So, the really the only place that they intersect is that they help you decide what to do. So, let's walk through a bunch of characteristics of generative research that we know and love, right? The goal is to sort of

love, right? The goal is to sort of discover a target audience or maybe understand their context. It happens in the early stages of the product development life cycle. Um, your

stakeholders are typically product managers, designers, product marketing, customer success. Uh, and it usually

customer success. Uh, and it usually takes you a few weeks to do. So, let's

contrast that with what I'm going to sort of talk about a bunch today, which is what I consider strategic research.

So, for one, your stakeholders are very different. you're talking to the head of

different. you're talking to the head of function, the board of directors, your executives. Um, it happens over the

executives. Um, it happens over the course of months and years. Like we're

we're tracking things month over month to see when the signals change and there's an inflection point, whatever it may be. Um, you're trying to understand

may be. Um, you're trying to understand market trends, you're trying to understand the competitive landscape, what are your organiz organizational goals, which direction should you go as a company? Um, and it's completely

a company? Um, and it's completely separate from the product development life cycle. So, a really really like

life cycle. So, a really really like easy hint is that if you're doing work and it fits into the PDLC, it's probably not that strategic. These things are

like completely divorced. Um, and I'll talk more about that. So, yeah, we use this this word research to refer to a lot of different things. Most research that we have done

things. Most research that we have done this far is frankly tactical research.

So, what do I mean by that? Like what is the definition of tactical? according to

Miriam Webster uh of or relating to tactics such as small-cale actions carried out to serve a larger purpose or things that are done

in like a sort of limited and immediate end in view. Let's reflect on that for a second. Like this is the dictionary. I

second. Like this is the dictionary. I

didn't make this up. This is the dictionary definition of strategic.

This sounds like most of the work that I have heard researchers talk about and most of the researchers that I've talked to like, huh, this is interesting. We

tend to operate on other people's timelines. We tend to do our work with

timelines. We tend to do our work with very specific goals in mind, often directed by other people. Uh, and we don't have a lot of like our own roadmap. I've I've had many

roadmap. I've I've had many conversations with researchers and research leaders about trying to define the research roadmap. Um, yeah, that's a thing that we all struggle with. And I

want to I want to note the title of the talk is not research is completely broken. We're really good at this. We've

broken. We're really good at this. We've

gotten really good as an industry at like hitting our marks, hitting our timelines, delivering the value that we promise. But I think we're, you know,

promise. But I think we're, you know, it's it's sort of holding us back. So

this is if this is tactical, what do we mean by strategic? Like what's the definition of strategic and what is strategic research? So we'll go back to

strategic research? So we'll go back to the dictionary this time. Cambridge

instead of Miriam Webster. Uh the

definition of strategic is relating to the way in which an organization decides what it wants to achieve and what it plans to do over time. So it's a very very different thing than this whole

tactical world. Javier Vargas uh

tactical world. Javier Vargas uh recently summed it up as tactical will tell you how strategic will tell you what. So if you are trying to figure out

what. So if you are trying to figure out like okay we know the problem we know the solution we just don't know how to do it. that's tactical work and tactical

do it. that's tactical work and tactical research. Whereas if you're like, "We

research. Whereas if you're like, "We don't even know what direction we're supposed to go." That is more on the strategic end. Um, yeah, we're treating these

end. Um, yeah, we're treating these things the same, but they're fundamentally different. So, there's a

fundamentally different. So, there's a few things we should rethink. First of

all, what are the timelines that we operate on? Who are the stakeholders?

operate on? Who are the stakeholders?

What rituals do we go through as researchers? And what are the

researchers? And what are the deliverables? What are we producing at

deliverables? What are we producing at the end of all this work? So I sort of mapped out this side by side. We're

going to walk through uh the horizon in which you operate on timeline scope etc. So first um if you're doing tactical work you're looking at like 0 to six month time horizon like what's what are

we doing in the next half a year or so whereas strategic work it's like what are we doing in the next three years and how is research informing what we as a business are doing. The timeline in

which you operate on short versus long right if your project takes a few weeks cool. if it's taking months at a time

cool. if it's taking months at a time and you're probably thinking, "Oh no, that sounds like a nightmare." Um,

there's ways to sort of get around it t being that uh rough. But yeah, if it's a timeline of months, if the scope is narrow versus broad, I'm not going to talk through each of these because you

all can read. But I think the thing that's really important to highlight here is like the deliverable is very different at the end of the day. Like we

do re work as researchers and you're asked to provide a recommendation.

There's no recommendation when you're doing strategic work. It's like, hey, here are all the things we could possibly do. Here are the opportunities

possibly do. Here are the opportunities that we as a business can pursue. Um, trying to do strategic work

pursue. Um, trying to do strategic work in this like tactical frame that we've built uh is kind of like to bring back this previous dance metaphor, it's kind

of like trying to do ballet at the club, right? Like these things just like don't

right? Like these things just like don't work and don't line up. and we're

wondering why we may be struggling to have the strategic impact that we, you know, think we should. So, you're

probably like, "All right, Brad, cool. I

get it. Like, why why should I care? I

don't know. I'm just trying to do my job, right?" So, um, I spent the past

job, right?" So, um, I spent the past year uh at Web Flow sort of shifting the research function in this direction. Um,

and things were starting to happen.

People were starting to take note. So I

was getting like DMs from exacts asking me like hey what what do we what do we have any research about AI or hey what have we learned about uh you know

agencies versus enterprise customers right so like the chief uh revenue officer be like Brad give me the summary of the last 10 research studies that we've done that's pretty cool we

actually for a while had a standing exec round table like a monthly meeting with a bunch of exacts where they're like cool tell us what you've learned here's some stuff that were thinking about. Um

it was a really cool opportunity. The

work that we were doing as researchers started to show up in like presentations to the board of directors and like in the discussions that they're having. Um

the stories that came out of research would show up in like marketing launches as like these customer sort of stories about the features that we were building and whatnot. Uh in short, we kind of had

and whatnot. Uh in short, we kind of had a seat at the table. However, I don't think that's the right metaphor. I don't

think we need a seat. I think it's our job to design the menu. It's our job to figure out like, okay, what what is actually happening at these discussions?

So, you're probably like, "Okay, awesome. I'm in. Sounds great. How do we

awesome. I'm in. Sounds great. How do we do it?" Um, this is my favorite slide in

do it?" Um, this is my favorite slide in the whole deck because I'm a troll. When I pitched this talk, it

troll. When I pitched this talk, it included how and Alec was like, "You can't do that in 20 minutes." Like, this is this is a whole other Maybe if I'm back next year, we can talk about how to

actually go through this. But the short answer is that you can't do strategic work until you have an answer to the tactical questions. So that is like

tactical questions. So that is like whether that's democratization, AI, whatever it may be. Um you have to solve those answers first because if you're like, hey, we're going to do this work that is 3 years down the road map and people have questions about what's

happening right now, they're going to think you're crazy. So you have to find a way to, you know, solve that. Moving

on, there's three meaningful pieces that I want to talk through. um sort of in the way that I have I've structured strategic research and this isn't the only way to do it but this is what we've

been doing at web flow and this is the way that I've encapsulated it there are inputs there are rhythms and there are outputs and I'm going to talk through each of these sort of one at a time but

inputs are basically everything that goes into this like highle planning exercise that we have rhythms are you know sort of where the magic happens it's like what we do how we work what

our timelines look like etc etc. And then the outputs are what we produce, like how our work impacts the three-year plan. Great, everyone got it. Cool. See

plan. Great, everyone got it. Cool. See

you at lunch. Kidding. So, yeah, we we'll walk through a bunch of these one at a time. What are the inputs to strategic research? Uh, there's a

strategic research? Uh, there's a variety of formal informal sources. So,

one of them is voice of the customer. I

talked about this a bit yesterday. Um,

Web Flow is using interpret, but basically you pipe a whole bunch of passive signals. And when I say passive

passive signals. And when I say passive signals, I mean support tickets. I mean

your like product reviews, threads on, you know, Reddit and Twitter and everything else. Um, yeah, you sort of

everything else. Um, yeah, you sort of put all that stuff into one place. And a

VC program lets you do two things. One,

you can get a monthly report to say, "Hey, here are the things that people are talking about." You can also do ad hoc queries, which is really cool. So,

highly recommend setting up a VOC program if you don't have one already.

Um, long-term bets are something that go into this this plan like what are the things that your business is working on and what's what is your company's roadmap that should obviously influence your road map as a research function. um

these questions from exec and if you don't have execs DMing you because I recognize not everyone does um look at look at the all hands meetings look at the uh the quarterly reports if you're publicly traded like what are these

things that what are the topics that they're talking about and how can you as a researcher answer and support those things um there are patterns in the requests that you get as researchers

like huh it seems like we have a lot of questions about this one topic that's probably because we don't know enough about fundamentally to make intelligent guesses. So if you're getting a lot of

guesses. So if you're getting a lot of requests, you're you're doing this tactical work, start to think through why, like what's behind that. And the

last one is my favorite one. Um some

good oldfashioned existential dread. Um

you have questions about your business, right? You sort of know what's

right? You sort of know what's happening. Uh what are the questions

happening. Uh what are the questions that you have? So yeah, let's zoom in on that real quick for a second. What are

you worried about in your business? Not

in your life. There's a lot of other things, but yeah, like what are the things that everyone else is missing? I

think researchers have this like unique perspective because we don't I often joke with people when they ask what I do. It's like, oh, I don't I don't

do. It's like, oh, I don't I don't actually do any work, right? I don't

build it. I don't sell it. I don't

support it. I don't design it. Like it's

our job to find problems and tell somebody else about them. So, that gives us a unique perspective. And it also we have a superpower, curiosity, right?

Like we are researchers. That's kind of what fuels us. So what are these questions that you have that no one else is talking about? Ask those questions.

It's kind of our job. We talked about at the research leadership summit yesterday like be brave, right? Sometimes you have to ask the question or report the data that people makes you unpopular. That's

kind of our job as researchers not to be unpopular but uh but to ask those questions. So we'll talk about rhythms

questions. So we'll talk about rhythms next. Um again these are like the things

next. Um again these are like the things we do, the cycles we go through. Um

again voice of the customer is important. These monthly reports are

important. These monthly reports are really important. You may be wondering

really important. You may be wondering why is it month over month? Because week

over week is too soon and quarter over quarter is too long. Like not enough movement happens on a weekly basis for you to really see a signal. And if you wait an entire quarter, you might actually miss something. So yeah,

there's a way to bubble up passive signal into this active domain because we all know we can't do all the research we want to do. So how do we decide what is the most important thing to work on?

There are, as I mentioned, these exec roundts, which I'll talk more about uh in a bit, but monthly meeting with execs, that's kind of useful. We do what I call multi-study research initiatives,

right? So, we look at one of these big

right? So, we look at one of these big problems, these big questions, and we plan out a handful of different research studies, individual studies that all tie together. It's going to take us months

together. It's going to take us months to do this. You probably want to keep your people up to date. You want to keep them engaged. These bi-weekly

them engaged. These bi-weekly stakeholder meetings uh and updates are really important one because you don't want them to get bored. You don't want them to like lose track of your research, but it also helps you adjust, right? As you're doing things month over

right? As you're doing things month over month, things may change in the business. So, you want to make sure

business. So, you want to make sure you're adapting to that. Um, and the last one is this metaanalytic step, right? like you do a bunch of research,

right? like you do a bunch of research, you prepare your findings and then every single time you do that, you update like the global knowledge about this topic so that you can at a certain point say okay

we've learned all these things and here's what it means. It's targeted per initiative. So whatever whatever the big

initiative. So whatever whatever the big you know multi-study research initiative is your meta analysis is relating to that. As I said I want to zoom in on

that. As I said I want to zoom in on this real quick. What happens in these conversations? What is this? It's really

conversations? What is this? It's really

as I said just an open discussion. Um,

and it's easily the most expensive meeting I've probably ever been a part of because who's there? You know, the chief product officer, the chief revenue officer would often show up, CTO, CFO,

um, leadership within the insights org.

Um, and then like special guests every now and then, whatever the topic is. You're wondering, that's a lot of

is. You're wondering, that's a lot of people, that's a lot of money. How do

you prepare for that? Put a deck together with everything that you've learned. And Web Flow does a lot of like

learned. And Web Flow does a lot of like pre-ereads. So record like a Loom video

pre-ereads. So record like a Loom video or something that basically says, "Hey, here's the things I want you to focus on. Here's what I would love to discuss.

on. Here's what I would love to discuss.

Here are the updates that are are relevant. It's not a status update. Do

relevant. It's not a status update. Do

not waste their time with that." Um, and you want to have this live discussion.

So you don't want to do this async. You

want to actually sit down with these folks and let them poke and prod and let you sort of answer the questions that they come up with. So the last thing here are

with. So the last thing here are outputs. I like this little pyramid

outputs. I like this little pyramid metaphor because they all kind of build on top of each other, right? So, as

researchers, we all produce data. We all

produce nuggets, right? The little like your little insight nugget. Um, you

know, we all do that. This is actually useful because it lets the product teams when they pick up these things eventually have the sort of the paper trail and they can do ad hoc queries and

analysis on this. It also plays nicely with the voice of the customer program.

So, yeah, make sure you're documenting your findings. A lot of us also prepare

your findings. A lot of us also prepare research reports, right? We say, "All right, did some research, here's what we learned." Um, and you summarize

learned." Um, and you summarize that. A lot of teams stop here. Like a

that. A lot of teams stop here. Like a

lot of people just like we did our report, we're done. There's more.

There's like extra steps you can take.

So the next one, as I already mentioned, is this like metaanalytic roll up of like, okay, we did a research study, we learned this thing. How does that relate to everything else that we know? And you

can do this without having a strategic initiative. You can just you can just do

initiative. You can just you can just do this with your research. Every time you finish a study, put it into some sort of global like executive executive summary.

What is what is what does this contribute to the overall knowledge that you have as a business? This is where you know if

business? This is where you know if you're doing meta anal meta analysis hard word um yeah there's another step here. So like okay we we've

summarized our learnings. What do you do after that?

You have to figure out whether or not this is worth investing in. So most

teams will not go talk to their pricing and packaging team, their product strategy team. You have to do some

strategy team. You have to do some market sizing. You have to do some uh uh

market sizing. You have to do some uh uh like maybe talk to an engineering leader and figure out what's the approximate level of effort it would take to support something like this because if you go to your execs and you say, "Hey, we want to

do this thing." They're going to look at you like, "Right, but why? Like what's

the what's the value? How what's it going to take? etc." So if you make it through that gauntlet, you get to a roadmap item. We actually take things

roadmap item. We actually take things from this project, this program, and put it onto the product uh backlog, which is pretty cool. So again, zooming in on one

pretty cool. So again, zooming in on one of these. How do you validate an

of these. How do you validate an opportunity? There's some familiar

opportunity? There's some familiar things that we all do as researchers.

Triangulation, right? Be a good mixed methodologist. Um a lot of talks on

methodologist. Um a lot of talks on Monday. Yeah, that was Monday growth UXR

Monday. Yeah, that was Monday growth UXR were about using logs, right? like how

to how to you know make sure that what you're presenting is you know as foolproof as it can be because execs are really good at poking holes in things and they really like doing that. Um but

some unfamiliar things market sizing uh yeah strategy pricing and pack packaging what's the total addressable market of this proposed feature. If you don't know what those things are, it's okay. But um

yeah, these are the things that we start to we start to think through. And if you present something to your leadership and you say, "Hey, users want this thing."

They're going to go, "Okay, whatever."

But if you say, "This represents a $20 million increase in um annual recurring revenue." They might go, "Okay, all

revenue." They might go, "Okay, all right. That's we'll consider that.

right. That's we'll consider that.

What's it going to take for us to build that?" 2 million. Cool. Okay. Yeah. So,

that?" 2 million. Cool. Okay. Yeah. So,

you kind of need to speak their language. And I know we probably heard

language. And I know we probably heard talks about speaking the business's language, but yeah. Um, I want to walk through an example real quick. So, Web

Flow launched this thing last year called analyze. Web flow is a website

called analyze. Web flow is a website building platform and they decided, hey, instead of having to go to Google an analytics or whatever, you can do it in web flow. Now, um, this happened before

web flow. Now, um, this happened before most of the planning happened before I joined the company, but I wanted to use it as an example of like what would this have looked like if we had gone through this strategic research process. So the

inputs would have been, you know, people are using Google Analytics and complaining about how complicated it is.

There are threads on Reddit and Twitter and they're saying, I really wish there was a better option. There's an internal sort of idea

option. There's an internal sort of idea of like as a company, Web Flow probably wants to capture more of the market. So

how might we do that? And another thing that Web Flow rolled out last year is that there's this evolution from being a website builder to being a website experience platform. So those are some

experience platform. So those are some of the things that go into this as input. The rhythms that you know we

input. The rhythms that you know we would uncover this need through these exact roundts that they want us to expand the market. Um we would share insights from the voice of the customer program that we're hearing more people

chatter about this and then you know we do a bunch of research and we do that meta analysis and provide those bi-weekly updates and the outputs are basically hey what sucks about analytics? What would it take to make

analytics? What would it take to make something better and how much money would that be worth to us? So to wrap things up here, just like

us? So to wrap things up here, just like not all dance is the same dance, not all research is the same research. It feels

really good in the moment to do tactical research because you get you get this instant gratification out of it, right?

You do some work and you present your findings and people go, "That's great."

And you're just like, "Nailed it." And that's that's okay. Like go

it." And that's that's okay. Like go

nuts, have fun, go jam out with your besties. However, there's a time and a

besties. However, there's a time and a place to do something more like Alvin Ali's Revelations, which is telling the story of faith, perseverance, uh, grief, you know, so you can go out

there and do something that is beautiful, impactful, and all inspiring as researchers. Thank

as researchers. Thank you. That was amazing. All right, we do

you. That was amazing. All right, we do I believe we should have some questions.

Um I am searching for our mic person who have a mic. Mic runner is coming now.

All right, I think we have a few spots around. So there a couple of hands up

around. So there a couple of hands up here and go for it.

Thanks Brad. Um great talk. I'm

wondering about the pivot that you've gone and maybe this is next year's talk but like from when you were younger and more junior to where you're at now where you are now entering into these con

conversations with these executive stakeholders with confidence being able to have that open dialogue whereas when you were more junior I mean like when I was more junior I would have been petrified to have that right and so

we're beginning to have some of that but we're like figuring out like we're finding our courage within that space. So, how did you find your

that space. So, how did you find your courage?

Yeah, it's a really good question. Um,

trial and error is certainly part of it.

The thing to remember when you're talking to execs is they're just people.

Like, yeah, they may have, you know, lots of responsibility and lots of experience, but like you can just have a conversation with them. You can just say, "Hey, like here's some thing I'm working on. Like, what do you think?"

working on. Like, what do you think?"

And most of the time if you're like afraid of talking to an exec, that's really just like your own insecurity because what's the worst thing that happens? They go, "No, that's not quite

happens? They go, "No, that's not quite right. Here, let me help you." Right?

right. Here, let me help you." Right?

Like exacts can be scary. Not everyone's

exec is like nice, but they're just people and you are all trying to accomplish the same goal, which is to like push your business forward and make the right decision and provide value to your customers. So, I don't have like a

your customers. So, I don't have like a magic bullet as to how to do it. I think

it's just like, hey, um, you just kind of have to practice, but yeah, they're just people at the end of the day. And

just have a conversation with them.

That's great. Uh, I think there's one actually one over here and then great. We'll get the mic to you.

then great. We'll get the mic to you.

Hi. Uh, great talk. My name is Corey.

Um, in the past I've been a contractor and I think Drillbit Labs last year did a study on job descriptions that more companies are moving into research as contractors and when I'm a contractor

I'm doing tactical work and I'm not even allowed to talk to decision makers structurally. Uh, so I'm curious for

structurally. Uh, so I'm curious for those of us here online who are contractors who are systemically set up not to do strategic research, what are

the first steps they could take?

It's a really good question. Um, it's

going to depend upon a lot upon the environment that you're in. Um, but I think I think the way you talk about your work can have a huge impact on the next set of work that you do. So, if you do your work, you present your research,

and you say, "Hey, here's what we learned." And you kind of leave it at

learned." And you kind of leave it at that, you're going to get a different reaction if you're like, "Here's what we learned, and here's like some rough market sizing, or here's here's how I think this might impact our bottom

line." Right? at the end of the day you

line." Right? at the end of the day you present research you're like this has the opportunity to increase our conversion rate by you know some percent. So I think even if you're stuck

percent. So I think even if you're stuck in like like pigeon hole in this very tactical space, you can start to think more strategically, think more about the business, think more long term. Say,

hey, we like often times when we present research that we've done. Um we'll say, hey, like what are some of the next steps? And one of those next steps is we

steps? And one of those next steps is we might want to investigate this other thing that we uncovered that, you know, wasn't really part of the original plan.

So I think there are ways to sort of flex in that direction um and have conversations outside of the context of presenting your work that are like hey I would love to do more work like this

what would it take how do I get there like recognize that you are hired for a certain thing but if you are interested in providing more value um most companies like like that from their

people.

All right another question back there.

Thank you so much. Good morning. Great

talk. U my name is Maria and as a former dancer myself, I really resonate with this analogy. Um I had a a follow-up

this analogy. Um I had a a follow-up question about some of the rhythms that you identified. I really love the way

you identified. I really love the way that you mapped that out and I was curious about your approach to these um executive roundts. In other words, you

executive roundts. In other words, you know, you specified that you're not coming with this kind of broad like here's what I've been doing, but to what degree are you coming with a specific

point of view and market opportunity?

Have you played around with the zoom um and the focus of those roundts and what works best for you? Yeah, it really depends where we are like sort of in the

life cycle of a project because I'm not going to do one set of interviews and saying this is how much it's worth like to the business, right? So, um, yeah, it it it sort of goes through its own

natural cycle where like, okay, we, you know, the way that it would work at Web Flow is that we would like go into interpret, we would go into the voice of the customer, pull a bunch of data, then we would do maybe a survey or some interviews or whatever, and we're

already starting to triangulate. So, if

you're in that very early stage, you can say, "Hey, execs, like here's what we've heard. Here's what we learned. Here's

heard. Here's what we learned. Here's

what we think it applies to our business." And at that point, they may

business." And at that point, they may say, "Great, go talk to pricing and packaging." like go start do some market

packaging." like go start do some market sizing or they may say you know what that's actually not even a priority for us anymore because something else is happening. So you'll like a lot of times

happening. So you'll like a lot of times they're like fists and fits and starts with these projects where you're like I'm going to do this thing. I'm going to work on it really hard and then you you get to your next round table and they're like oh cool yeah um that's not

important anymore. We're going to can

important anymore. We're going to can you do this other thing instead? Yeah I

think that's I think that's about it.

All right we think I have um just one more question. And I think there's

more question. And I think there's there's one up here just in the second row.

Thank you. Thank you, Brad. Uh my name's Lucy. Hi. Um what was the biggest hurdle

Lucy. Hi. Um what was the biggest hurdle from transforming your uh research organization from a tactical department to a more strategic one? Were there

people who questioned the like maybe for the first time that you did this, were there people who questioned the value of diverting your time away from the tactical stuff? Thank you. Yeah. Um,

tactical stuff? Thank you. Yeah. Um,

there are people that still question it because you have to make this fundamental trade-off. So, as I like

fundamental trade-off. So, as I like alluded to, it's like you've got to find a way to answer those tactical questions to have any sort of bandwidth for this.

Um, we did that through heavily democratization. Like we we I I did it

democratization. Like we we I I did it at all zero, we did it at Webflow as well. Like really heavily invest in

well. Like really heavily invest in building out DIY research, democratize, whatever you want to call it, lightweight research at all zero. You

always have to brand it a certain way.

um because the word democratization is evil now.

Um but yeah, like we did that through providing a way for PMs and designers and folks to answer those questions so that we can start to like buy ourselves a little bit of leeway. So it's it's

still like when you are telling your your peers and stakeholders like, "Yeah, I'm actually not going to do that thing you're asking me to do because there's this other thing we're going to work on." That's a tough cell. Like that is

on." That's a tough cell. Like that is just a difficult thing to do. So it's I found success not in saying just we're not going to do that. It's actually like here we're going to help you. You can

here are ways you can answer those questions and there are quite literally times where we're working on something that is more strategic and for whatever reason something becomes urgent and we

have to again pause what we're doing over here to step in to help out. So

yeah, it happens all the time.

Those are great questions. Thank you so much for asking them. Brad, thank you so much for that marvelous talk. Let's give

a warm round of applause, Brad. Thank

Brad. Thank you. All right. One down, seven

you. All right. One down, seven brilliant more to come. Hey. All

right. Now, for the next talk, what's been running around my mind since sort of heard about this and talked to Alec about it and just been excited for what Zoe's is going to share is let it go.

Let it go. Do we care about the itty bitty things in research a little bit too much? And so Zoe is going to help us

too much? And so Zoe is going to help us today to care less. Please give a warm round of applause to

[Applause] Zoe. Brad and I made very different

Zoe. Brad and I made very different footwear choices, so I don't think I have the same movement he does. But come

on, guys. Jimmy Chu, shout out 60% off. Not to brag. Um, thank you so much.

off. Not to brag. Um, thank you so much.

My name is Zoe. Um, I'm a researcher at Google and I know we've heard some amazing speakers across this week giving you a whole bunch of things to care about and do sometimes, but give fewer

[ __ ] the rest of the time. And Alec,

I'm so sorry. I know I promised not to swear, but I made it almost a full 30 seconds. So, why do you want to care

seconds. So, why do you want to care less? Why am I asking you to do this

less? Why am I asking you to do this kind of revolutionary thing? I have

heard so many questions this week of how do I get out of this space of defensiveness? How do I prove to this

defensiveness? How do I prove to this other person? We have become in this

other person? We have become in this antagonistic role with our stakeholders and it's time to get out of it. Now, I

want to be so careful. I am not asking you to avoid care. Care is still tantamount and so important to what we do as researchers. It's central to our success. It's central to what we do in

success. It's central to what we do in our jobs. I am certainly not advocating

our jobs. I am certainly not advocating for you to be careless. I am actually a bit of an [ __ ] and if your P values or effect sizes don't make sense, I will be the one commenting you in your deck.

That is not what we're getting at here.

What I'm asking you to do is to be significantly more judicious with your care, how you choose to apply it, how you think about yourself, your products, your team, and your life.

Now, one of the things we're going to get at here, because I've been given this advice a lot in my life, and it didn't resonate for a long time, is there is a difference between care and control. And these are the two things we

control. And these are the two things we want to make sure that we're balancing.

Care does not equal control. No matter

how many books Gwennneth Paltro makes a million dollars selling, your passion will not bring things into being. It'll

be a part of it, but that is not your responsibility. So, how are we balancing

responsibility. So, how are we balancing the two? That's what we're talking about

the two? That's what we're talking about today. Now, I know often people give you

today. Now, I know often people give you these wonderful little things about why they in particular are so expert at this. Now, my former head of research is

this. Now, my former head of research is sitting here, Cat Murray. Um, and she's probably going, "Oh, this is going to be a funny talk from this woman." Why? Same

as my colleague. She said to me when I told her about this talk, "Lovingly, you care more about everything more than anyone I've ever met." This was three months ago. I told my therapist about

months ago. I told my therapist about this and she said, practically frothing at the mouth, I couldn't come up with a better project for you of my life dependent on it. This is something I

struggle with and I still struggle with.

But this is so important right now, especially at a time where we are fighting for meaning, fighting for influence, and in so many cases fighting for our jobs. And our tendency is to

take on more, not less. And I'm going to tell you today, you got to start taking on less.

So, what does anyone do that's trying to now figure out how to be chill, cool, fun? I interviewed 24 people about this

fun? I interviewed 24 people about this across auto tech, PR, and nonprofit industries. I had an amazing talk with a

industries. I had an amazing talk with a death duel about this. Blew my mind.

Happy to talk about that more afterwards. And then I said, well, what

afterwards. And then I said, well, what else can I do? And I read every piece of public litter I could sure I could find to really understand, well, how does this come to be? How do we give this advice, right? What do we do with this?

advice, right? What do we do with this?

I know it's a very reasonable approach, but I think it played out. And really,

the central tenant is how we're going to make this connection between care and control. Right now, it feels messy. It

control. Right now, it feels messy. It

feels opaque. And we're told often that our passion is the differentiator. So,

how do we get to this place where we can still have this passion, but we are not completely responsible for it? How do we draw a better line between care and

control again so we can use that care judiciously? So here's my radical

judiciously? So here's my radical challenge to you. I want you to actually care less. In a world where we are told,

care less. In a world where we are told, "What other hat can I wear? What other

title can I grab and dawn as a coke? How

can I prove I am worthy every day?"

Stop. Care less. Confidently acknowledge

what you can and cannot own. which you

can and cannot influence, but let go of the rest. The idea that we can put

the rest. The idea that we can put effort into every little bit of that in some hope of building a care Lego tower, a wall of protection is just not true.

So, let's attack this problem together.

Now, one of the things I want to get at is what are we campaigning against? How

did we get here in the first place? I'm

arguing that there's three things.

Reactiveness, that we are responding to other people. Other people have an idea

other people. Other people have an idea and it is our job to advocate for the user against that monolithic value. Now

we're seeing this a lot with AI and democratization. How are we defining our

democratization. How are we defining our success is far too narrow. And finally,

perhaps the hottest take, dangerous empathy. If anybody's been in a

empathy. If anybody's been in a relationship for more than a couple of weeks, somebody has said to you, you're not responsible for their happiness.

Right? And that's really important. But

by the way, you're not responsible for anybody else's empathy either. And I

know this shows up in career ladders and expectations, but you cannot make people care about something. You can invite them to, but you can't force them to.

So, let's draw again that connection between care and control. Why do we care so much? A

control. Why do we care so much? A

couple things here. I think one of the biggest is how insecure we are in our own value right now. And let's be real for a second. Some of that is true.

Often research is the third or fourth step down before we get to the biggest tables in the room. We're not

necessarily there. Our paychecks might be smaller than our peers. Certainly the

number of us is smaller than our peers.

I have a ratio of 1 to 11 PMs. 1 to three designers, not so bad, but 1 to 11 PMs, right? Am I valuable? [ __ ] yeah.

PMs, right? Am I valuable? [ __ ] yeah.

Sorry, Alec. I'll do better, guys. But it's hard to see that on

guys. But it's hard to see that on paper. often this really does put us in

paper. often this really does put us in this against you are not listening to me. You are not taking me seriously and

me. You are not taking me seriously and that hurts but it's also working against us. So we want to work into an

us. So we want to work into an enablement mindset versus a protection mindset. Now another big piece of this

mindset. Now another big piece of this is how we showcase our value leaning into that monolithic value. We have a tendency to say what is it that I uniquely have that no one else has. But

our peers aren't necessarily asking this question. Right? This leads us to saying

question. Right? This leads us to saying things like, "We cannot do AI. We cannot

do democratization. Only research can do this thing." And we already know that's

this thing." And we already know that's not true. Come on. These companies all

not true. Come on. These companies all around us are doing great work. They're

empowering us to take on projects that we actually want to take on, like Brad said. And let's also be very clear, none

said. And let's also be very clear, none of us is well staffed enough to be able to take on all those questions even if we wanted to. So, how do we use that care and control balance to make sure

we're taking on the right things but not everything? That also leads to a

everything? That also leads to a tendency to double down on the status quo because it might not be working very well as Brad told us, but we understand it and what's nest feels really scary

and that's okay. Care a little bit less and let it go. And finally, we have a tendency as researchers to focus on intangibles. We are extremely reliant on

intangibles. We are extremely reliant on empathy. It was only very very recently

empathy. It was only very very recently that Google removed from their leveling that we are responsible for the empathy of other people. You can drive empathy.

I can make you hopefully think about something but ultimately I can't make you do anything about your emotions and we can't put that on a researcher because by the way at the end of your

perf or performance rating I'm not going to go to cat and go oh but look how much more empathetic they are. Right? That's

not going to work. And then we tend to also lean into immeasurable things that again at the end of those performance ratings we're like well our name didn't make it on that document so I guess I don't know if I had an impact I don't

know if I actually built something but we know we did so we need to step back from that. This is all to say your care is

that. This is all to say your care is your most powerful resource and you need to start using it judiciously. Now again

I'm not saying mail it in. I'm not

saying halfass it. But I'm saying be deliberate in what you take personally and what you take responsibility for.

And it can't be everything. Now, how do we start to do

everything. Now, how do we start to do this? I've developed this really fun

this? I've developed this really fun little matrix for you. Very easy to adapt, I hope. Hope. Care versus

control. Low control, high control. Low

care, high care. First

up, also the designer changed these emojis and they're so much better. Um,

low care, low control. Hopefully, you're

not doing any of these, right? If you

don't care about something and you can't impact it, don't invest there. There are

so many things to do, skip it. Like, are

there times you're going to be forced here? Sure. But like 5% of your time,

here? Sure. But like 5% of your time, maybe. And it shouldn't matter, right?

maybe. And it shouldn't matter, right?

You're like, whatever. Do whatever

you're going to do. Next, high control, low care. I can do a lot about it, but I

low care. I can do a lot about it, but I don't care about it. That's maybe 20% of your time, right? You're going to do some of that research. You're still

going to have strong impact. This might

be that like pure usability that people are talking about. Doesn't have a lot of impact. Doesn't have a lot of there, but

impact. Doesn't have a lot of there, but you can make it happen and generate things for it and make sure that button isn't blue. The area that gets really

blue. The area that gets really dangerous is where we love to be and why we are so antagonistic, which is I don't have any control and I care a lot.

Right? This comes up so often with some of the work that we think is most important or at least I do. But it

doesn't mean that you can't take these on at all. But it means that it might not make sense to consistently choose the research project that brings you into frustration, but rather think about

how you can use things as a side project. Other approaches to this where

project. Other approaches to this where yes, you're still going to feel fulfilled, but you're not going to be coming to work every day going, "God, I hate this and nobody cares about the

user and nobody," right? This lack of control feeling. We want to get out of

control feeling. We want to get out of that. And finally, the area we really

that. And finally, the area we really want to hang out. High care, high control. I care a lot and I can do

control. I care a lot and I can do something about it. Now, again, we have way more projects than we can ever ever do. So, you have a lot more influence to

do. So, you have a lot more influence to make sure you're picking the ones that matter for you. Again, spending about 75% of that time in that high care, high control. Another 20% in high control,

control. Another 20% in high control, low care, as long as it makes sense. But

really avoid the other two as much as you can.

Now, the first time I got this advice, well, let's be real. I've gotten this advice since high school. It was

actually on my report card. I did not put it up there because it had too much text. Thank you, mom, for holding on to

text. Thank you, mom, for holding on to these for this long. But I remember saying, "Great.

long. But I remember saying, "Great.

What do I drop?" And they went, "Oh, I don't know. You choose." And I was like,

don't know. You choose." And I was like, "No, no, no, no. What's next? How do I know what to drop?" And they said, "Well, I don't know either. Good luck."

And I was livid. I was like, "This isn't helpful." So, I changed managers. didn't

helpful." So, I changed managers. didn't

solve my problem, FYI. So, what can we do to navigate

FYI. So, what can we do to navigate through that? What are some principles

through that? What are some principles here? Here's your first one. Define your

here? Here's your first one. Define your

success upfront, not your data. We have a really big tendency as researchers, again, monolithic value, to double down on our

data. Your data, as Brad pointed at,

data. Your data, as Brad pointed at, kind of doesn't matter. It got you there, and that's really important. But

the outcome, what you're driving towards, what you are building is what actually matters. Given that ABC, we

actually matters. Given that ABC, we must do this other thing. What are you moving towards? And you want to align

moving towards? And you want to align with people as you go. What are we building towards? Right? We can

building towards? Right? We can

interpret this plot in a bunch of different ways. Where are the data

different ways. Where are the data clustering? Where are these things

clustering? Where are these things growing? But you want to make sure

growing? But you want to make sure you've had that conversation with your team of are we building gold? Are we

making Willy Wonka's chocolate bar? Are

we building a Northstar? Because

otherwise, you get to the end of these projects and you go, "It is so clear to me that we should ABC." And they do something else and you're like, "I didn't get listened to. I got ignored."

Did you? Maybe. But more than likely, you thought you were building Willy Wonka's chocolate bar and they thought they were building a Northstar. And the

way they interpreted your data, because again, you gave them data, not findings, is that they got to take it the way they wanted. and it might actually be a

wanted. and it might actually be a perfect reflection of what you did. They

might be really surprised to hear the feedback that you didn't feel listened to. So, we want to upfront define that

to. So, we want to upfront define that success empathetically. Which means I need you

empathetically. Which means I need you to stop asking the question at least to your stakeholders, is this good research? And instead start asking the

research? And instead start asking the question, does this drive us forward?

You should still be asking the question, is this good research to yourself and potentially your manager, but not to your stakeholders? This means I'm going

your stakeholders? This means I'm going to challenge you for the rest of the year not to share a script, not to share survey text. They don't control your

survey text. They don't control your data. You should have a really clear

data. You should have a really clear sense of what that's driving towards. We

are here now and we are trying to get here. And the gap in our understanding

here. And the gap in our understanding between those two things is these four things. And then you as a researcher, as

things. And then you as a researcher, as the expert, are going to take that and figure out the method and give them those answers. Right? This is where

those answers. Right? This is where we're driving. Do not give up the rest

we're driving. Do not give up the rest of the control that you do have on those individuals. So to put it more simply,

individuals. So to put it more simply, crystallize that win. Talk about how you're going to get there. Right? We are

going to be aligned throughout this process. And then be obnoxiously loud

process. And then be obnoxiously loud about it. Right? It should be getting in

about it. Right? It should be getting in those emails, in those chats, talking to everybody, working with everybody. Next, relationships matter

everybody. Next, relationships matter most. We heard some great work on this

most. We heard some great work on this from Gus earlier this week. If you

weren't at growth UXR, I heard they had a great moderator, but definitely go check out his talk. Research and communication is so

talk. Research and communication is so important. Nothing is purely product.

important. Nothing is purely product.

Nothing is purely technical. Nothing is

purely a best path. It all exists within the context of the relationships with the people that we work with. You need

to communicate and you need to lean in.

Focus on connecting your findings to what different people are thinking about and how you can adapt. Again, if you're aligned on that northstar, you're going to have a clear sense of that. This is

how you are going to care strategically.

I am going to care about the things that the people I care about care about, and I'm going to bring in and fold in the things that I know are important. So, I

have control and care in tandem. Again,

you are not responsible for the empathy of others. I don't want you to go feel

of others. I don't want you to go feel like you have to make a best friend so they they feel bad if they don't listen to you. That's not fun and it's not

to you. That's not fun and it's not cool. But, it is important that you have

cool. But, it is important that you have a really good sense of what they're looking at.

Which means you're going to have conversations like this. What's keeping

you up at night? What's going to make us fail in five years versus tomorrow? How

do these things connect? And it's not necessarily connected to a single project. I have this conversation with a

project. I have this conversation with a couple of my directors at least once a month just to hear what they are thinking about today. There was a question before of like how do you know when it's okay to talk to these people?

They love talking to researchers. like

you're about to feel like the coolest person in the room because they're like, "Oh, I have a line in that they didn't know existed." So, go talk to them. Now,

know existed." So, go talk to them. Now,

this also means, and for those of you know, I used to be a wildlife biologist, so like didn't have to talk to humans at all. It was

all. It was great. This also means caring about

great. This also means caring about their lives outside of work and them caring about yours, right? Ask about

their kids. Ask about their rec softball game. Ask about their travels. You don't

game. Ask about their travels. You don't

have to know everything about them, but the more that we can connect to each other, the better.

Next up, start measuring stuff. You'll

see this is all over the care and control spectrum. It is really important

control spectrum. It is really important that we measure one way or the other.

There's more than one way to skin a cat, but only one way to know if you did it well. Right? You're going to make best

well. Right? You're going to make best friends with your data scientist and your PM because you're going to make sure that you see what's happening now and next. You can't control every launch

and next. You can't control every launch every time, but you can ensure whether or not you thought it was a good idea or a bad idea that it's being measured appropriately so success

continues. This should be a fundamental

continues. This should be a fundamental part of your work. Partner with all of these people if you have it. QXr is

quantr by the way, but these are the people that can help you get these measurements. I'm going to share a

measurements. I'm going to share a really brief story because I'm totally overtime from Lyft where we launched this new pricing tool within the Lyft rentals program which again I know you all loved. It was great.

all loved. It was great.

RIP.

Um, but when you popped to the page, we had a debate. Do we show you the cheapest car at every lot or do we give you more information? And I was like, h, control information for sure. This is

comparing an SUV to a minivan. This

doesn't make sense. There's too many factors here. But it tested pretty well.

factors here. But it tested pretty well.

We had good bookings. So, how do I push back on that? Right? It's clearly not a battle I'm going to win. What are the user experiences factors to consider there? How many people are changing

there? How many people are changing their car at the lot? How many people are rating lower satisfaction? How many

people are cancelling? Right? Push on

these other pieces so that in a month or two you can come back and say, "Did that make sense? Maybe bookings went up but

make sense? Maybe bookings went up but end fulfillment didn't." Own that conversation. Just because the decision

conversation. Just because the decision has made doesn't mean it will stay that way forever. Next, navigate systems. A lot

forever. Next, navigate systems. A lot of what we do sucks. Let's be real. Like

you did some great work and then you're like, "Well, the project just went away.

What happened?

We got to navigate those tides and roll with them sometimes. There's three

things here. You're going to lean on those relationships when you can and when it feels good. You're going to walk away from that project if you need to.

You are going to invest that care in yourself sometimes. And sometimes you're

yourself sometimes. And sometimes you're going to let go of that care and just roll the dice. This isn't what I think is right, but I'm going to see what happens. Now, I hate this because it's

happens. Now, I hate this because it's out of control. So, this is a script that works really well for me. I think

I'm missing something. This is when you're like, this is divorced from my understanding. My understanding of goal

understanding. My understanding of goal is X and what we're doing is Y, which is when you're saying this makes no sense. Can you help me fill in the gap?

sense. Can you help me fill in the gap?

So, why does this script work? I think

I'm missing something. We've talked a lot about antagonistic. This blows that up, right?

antagonistic. This blows that up, right?

I need your help. Can you help me for a second? Right? People are like, "Oh,

second? Right? People are like, "Oh, you're not disagreeing with me. You're

truly trying to understand. My

understanding of the goal is X and what we're doing is Y." Right? It is up to you now to fill that in. Pause here. I

hate this part. obviously love to talk.

Give yourself a second and let the other person really say where they agree or don't agree. And then you're going to

don't agree. And then you're going to point out whatever that gap was. Can you

help me fill it in? And they might go, "Oh, shoot. I misunderstood something."

"Oh, shoot. I misunderstood something."

They might go, "Yeah, that director really wants that thing." And you're going to at least be able to navigate the system and understand that your care isn't going to adapt it. But own

that. Next up, embrace imperfection. You know your 100% goal,

imperfection. You know your 100% goal, right? But if you get 20% of the way

right? But if you get 20% of the way there, you're 100% further than you were. So it's okay that you didn't get

were. So it's okay that you didn't get there. Give yourself grace in doing

there. Give yourself grace in doing that. Progress over perfect. And more

that. Progress over perfect. And more

than anything for researchers, I need you to let go of fault. I hear this with us all the time because we have made ourselves responsible for the user and the user alone. We are fighting battles

that no one else is fighting with us.

Own what you can. If something went wrong, great. That is you to figure out

wrong, great. That is you to figure out and to go forward. But you don't get to own every domino it hit along the way.

You're going to let go of that. And finally, I want you to

that. And finally, I want you to diversify your identity. I love this quote. Much as an investor benefits from

quote. Much as an investor benefits from diversifying their investments, we too benefit from diversifying our sources of identity and meaning. Yes, this is a little bit of a fluffy one. Alec wanted

it out. I wanted it in. And here we are with control, baby. You're going to increase your

baby. You're going to increase your self-complexity, right? Diversify the

self-complexity, right? Diversify the things that keep you powerful. Allow for

seasonality. I moved to Indiana for two years. I defined myself as a cyclist.

years. I defined myself as a cyclist.

That didn't go well. Give yourself some flexibility in that. And then if you've never done a value assessment, I want you to go home today and do one.

Identify your values because it comes so much clearer. I literally have them

much clearer. I literally have them taped to my monitor and somebody will give me some criticism that I don't agree with and I go, "God, does that track to my values?" No. Cool. Bye.

Right. Make sure it aligns. Just because

somebody has feedback doesn't mean it's good feedback, right? The more varied and interesting your life is outside of work, the more ability you do have to roll with the punches. When the tides go against you, when things don't feel

right, it all feels smaller because it's more clear that it is smaller. Now, how do you do this? I

smaller. Now, how do you do this? I

freaking love reading, so please try that. The good enough job and master of

that. The good enough job and master of change were two for me recently that were really powerful. You're going to dive into new things. If that's AI, if that's taking Airbnb experiences,

anything new, something to like get your brain moving, you're going to invest in your friendships, talk to people here today. There's a lot of folks that are

today. There's a lot of folks that are based in the same areas. Connect. There

is nothing better than connecting with each other. Mark Zuckerberg is about to

each other. Mark Zuckerberg is about to try and make a bunch of money off of getting you to talk to an AI agent instead. Don't do it. Make friends with

instead. Don't do it. Make friends with the people you're both in adorable outfits that like play off of each other. Become best friends. And finally,

other. Become best friends. And finally,

if you're able travel, it's one of the best things we can do according to the literature. This doesn't actually mean

literature. This doesn't actually mean hopping on planes somewhere else, though. Hey, head to Kenya. It was one

though. Hey, head to Kenya. It was one of the coolest places I've ever been.

But it like go try new restaurants. Go

try new things. Just expose yourself to novelty. At the end of all of this, life

novelty. At the end of all of this, life is the product. Life not product is the end goal. Right? One of the things I

end goal. Right? One of the things I heard that I think was perhaps most alarming from a lot of the most senior people I talked to is really what happens when we care too much is pretty scary. We want to make

sure we get so far from that and so empowered away from that that we feel really good. And finally, your care is a

really good. And finally, your care is a resource. Use it judiciously. Thank you.

resource. Use it judiciously. Thank you.

Zoe, that was wonderful. Really clear,

refreshing at the same time. Thank you.

We have questions. So, um, feel free to raise your hand and we will find you with the mic. All right, we're going to head over

mic. All right, we're going to head over here first and we will come back to you.

Hi, Zoe. Thank you for this talk. Always

learning from you and this was, you know, a great example of that. Um, love

the quadrant. want to do more high care, high control. What's an example of what

high control. What's an example of what that looks like and what happens when it doesn't align with business needs? How

do you like take that into consideration? Totally. So, I think one

consideration? Totally. So, I think one of the big things here you end up noticing is truly you have so much going on across your product spaces that there is almost always an opportunity to influence. Often the trap we get stuck

influence. Often the trap we get stuck in and Corey kind of mentioned this earlier is they've already decided what to do. That means you have low control,

to do. That means you have low control, right? You're doing iterative research

right? You're doing iterative research on something they've already settled. Give them another way to do it.

settled. Give them another way to do it.

Give them Rally or Susie or whoever else is in this room, but let yourself step away with the democratization. Let

yourself step away with the AI. You

actually have a lot of control in that when you're going, the risk here is really low and controlled and I can give them those next steps and it's going to give me this other freedom. Now, Brad

had gotten a question before of like how do you kind of think about these bigger projects in that space? It's going to inherently give you that space to do that and you're just going to be the one

that starts to bring it up. Right? I see

the business goal because we've aligned is chocolate bar. I know you don't see the connection to this right now, but here's where that connection is. Here's

that future opportunity. Here's how it ties these things together. It becomes a lot about narrative and storytelling.

That's great. Next question. Hi there.

Good morning. I'm Animeary and I I love this talk. I've had to learn quite a bit

this talk. I've had to learn quite a bit in similar circles of just care less.

Um, I'm kind of curious because this is a tough scenario and you can say no if it hasn't happened to you, but have you ever been in a situation where you're

under low control, high care and the care is due to users potentially getting or some users, some subset of users or

even non-users getting into might get into trouble or ethically kind of gray things where it is yeah the care from there might be users that might get hurt

or incredibly disadvantaged or any of that gray area stuff. And what did you do in that very difficult situation where your rights low control but high

care because there might be something ethic ethically involved. Totally. So I

I have been in these situations. I

actually work in privacy and security so it's kind of constantly top of mind.

luckily work with a really stellar group of people. But in a previous position, I

of people. But in a previous position, I was actually in a role where we were using facial recognition technology as a means to gain access to another product.

And we were told by the company that we were contracting out with, don't you worry, it's got an 85% success rate.

Nice. Okay, not bad. Well, where's that 15%. Right? What is it failing on? Um,

15%. Right? What is it failing on? Um,

by the way, it's people of color. We're

pretty good with white people and that's it. And then that becomes actually an

it. And then that becomes actually an unacceptable risk, right? because we're

actually segregating our users. We're

creating a worse experience for this specific group and that is not okay.

Now, business-wise, there was a strong need of like, but we want to launch this now. You might not have as much control

now. You might not have as much control within your own team. Those are the times to bring in other resources. Go to

your legal team. They will shut that [ __ ] down so fast, right? Bring in those other resources. Control the pieces that

other resources. Control the pieces that you can. Again, you can't control the

you can. Again, you can't control the business outcome. You can't get in that.

business outcome. You can't get in that.

And that's where researchers, we have this tendency to be like, "This is so insane. Of course, you're going to do

insane. Of course, you're going to do the right thing." But it's not because to them there's these other pieces they're building towards. So think about whose goals you are aligned with, whose control you would have, and step into

that as much as possible. The other

thing really is those metrics. If truly

you can't make that change, think about ways to measure it so it becomes clearer, especially with these kind of intangible things. Are people sticking

intangible things. Are people sticking around? Are we creating inequalities in

around? Are we creating inequalities in our product? How do you know that is

our product? How do you know that is happening? And then bring it up in a

happening? And then bring it up in a month and say, "See, it is happening.

We're gonna have this conversation again." When you have more control

again." When you have more control because you have more of that data.

So legal.

[Music] It's definitely when you want for online it was like will that make enemies going to legal? That's up to you for what

to legal? That's up to you for what that's important. In this case for me,

that's important. In this case for me, it was totally worth it. And actually

once they heard it, they were like gh you're right. Actually the other I used

you're right. Actually the other I used to work in PR. what we think of this looked at the front page of the New York Times that usually gets people to back down on their own is a is a pretty easy one. Um, but ideally you're in this

one. Um, but ideally you're in this relationship again where with that script. I'm missing something here. I

script. I'm missing something here. I

see this as a huge risk. Do you not? And

you'll get more of their understanding and come back around. Yeah, I think we have time for one more question. I think

there's one over on this side at the back over there. Thanks.

Hi Zoe, excellent talk. Thank you so much. My name is Annabelle. Uh I wanted

much. My name is Annabelle. Uh I wanted to ask a little bit about your steps for dealing with, you know, maybe you've aligned with stakeholders, maybe you've talked to them throughout the process

and then you're in this spot, you've gotten to a high care point and you get told no, we can't do anything about that. What are your first steps? How do

that. What are your first steps? How do

you follow up? Can you dive a little bit more into that? Yeah. So, this happens a lot and I think especially I work in a very engineering heavy piece right now.

So, the feedback will be like well but it's really hard to do to which I'm like you make so much money figure it out. Um

you can't say that part aloud at least to the engineers. But it is having that like

engineers. But it is having that like put your research hat back on. Why? Why

can't we do it? Well, how do you think it tracks without this? Like put that genuine curiosity. And sometimes what

genuine curiosity. And sometimes what you're going to end up noticing because look, people are not great communicators. The most books that I

communicators. The most books that I read are on like better communication.

And the better you get at it, the more annoyed you're going to be with everybody else who's not very good at it. But just keep pushing that. Hey,

it. But just keep pushing that. Hey,

what's going on here? What am I missing?

Where is this coming from? And usually

you're going to get a pretty good answer eventually. Even if you don't agree with

eventually. Even if you don't agree with it, you're going to understand it. And

like, let's be real as researchers, what drives us the most nuts is when we don't understand why this crazy thing is happening. And then again, lean back

happening. And then again, lean back into those metrics. Make sure you have a way and think through like grab your design partner. How do we know this

design partner. How do we know this isn't working right? When we think about kids success in school, it's not just grades. We're thinking about how many

grades. We're thinking about how many days a week that they're actually showing up, how many sick days are they taking? How often are they going to the

taking? How often are they going to the office? These other things. Think about

office? These other things. Think about

what those counter ex uh counter metrics might be so that you can bring it around even if you can't bring it around right now. But lean on those relationships.

now. But lean on those relationships.

Fantastic. Well, Zoe, this has been brilliant. Thank you so much for that

brilliant. Thank you so much for that talk. Thank you.

talk. Thank you.

Thank you so much. All righty. Well,

we're on a break. We're going to take a break. Amazing. Going to be able to like

break. Amazing. Going to be able to like digest what you said and what Brad said.

Come ask questions and uh we'll be back here online as well. 10:45. So, speak to some people, grab some coffee, make some friends. Make some friends. Thank you

friends. Make some friends. Thank you

all. Thank

[Music]

you. Heat. Heat. N.

you. Heat. Heat. N.

[Music]

Heat. Heat.

Heat. Heat.

[Music] [Music] Woo!

Yeah! Yeah.

[Music]

[Music]

Heat. Heat.

Heat. Heat.

[Music]

[Applause]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

[Music] [Music] [Music] [Music] Hey,

[Music] I feel hey

[Music] Hey.

Ooh. Hey.

[Music] [Music] [Music]

[Music]

[Music] Hey hey hey.

[Music] Heyoo. Hey. Hey.

Heyoo. Hey. Hey.

[Music]

Hey bro.

[Music]

[Music]

Down. Hey. Hey.

Down. Hey. Hey.

[Music]

Hey, hey, hey. Heat. Heat.

[Music] N. Heat. Heat. N.

N. Heat. Heat. N.

[Music]

Hey hey hey.

[Music] Hey hey hey hey hey hey hey hey hey hey

hey.

[Music] Heat.

[Music] Heat. Heat. Heat.

Heat. Heat. Heat.

[Music] N. Heat. Heat. N.

N. Heat. Heat. N.

[Music] [Music]

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music] [Music]

In a work world that's more disconnected and remote, how do we as researchers build trust? How do we get our partners

build trust? How do we get our partners to understand that our research can help teams reach new heights? Well, we know it's not through PDFs and presentations.

Even the most thorough research isn't enough to build that trusted connection to get you a seat at the table. What

builds connections is stories, experiences and feelings. Feelings you really get when

feelings. Feelings you really get when you're there. When your stakeholders see

you're there. When your stakeholders see you uncover a great insight live, they see the magic of research. And they see you. Because when it comes to research,

you. Because when it comes to research, seeing is believing. Turn research

skeptics into research champions with Look Back.

[Music] User interviews is the fastest and easiest way to recruit participants for research. We're known for fast matching,

research. We're known for fast matching, a highquality audience, and a simple workflow that researchers love. Start

for free at newsinviews.com.

[Music]

[Music]

Join a global industry of researchers, designers, and product managers who use Maze to get insights at the speed of

product development.

[Music]

[Music]

Oh yeah.

[Music]

Hey hey hey.

[Music]

feeling.

Feel Heat. Heat.

[Music]

Oh [Music]

yeah. Hello.

yeah. Hello.

[Music]

Hey, I got feel.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

[Music]

[Music] You want

me to be the only I every

[Music]

[Music] Hey hey hey.

[Music] Hey,

hey hey.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Hey down.

[Music]

Heat.

[Music] Heat.

[Music]

Yeah down.

[Music]

[Music] [Applause]

[Music] Yeah. Hey. Hey.

Yeah. Hey. Hey.

[Music]

Yeah. Yeah. Yeah.

Yeah. Yeah. Yeah.

[Music]

Yeah. Yeah.

Yeah. Yeah.

[Music]

[Music] Black.

[Music] [Music]

Hey hey hey.

[Music] [Music] Hey hey hey hey hey hey hey hey

hey hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey

[Music] hey. All right, good people. Especially

hey. All right, good people. Especially

those of you outside. I'm going to get you to start making your way back. Hop

into your seats. Wrap up those sponsor combos cuz we're about to get going in the next minute or so.

[Music]

[Music]

[Music]

Folks, I'm going to get you to grab a seat, wrap up some conversations, cuz we have some great sessions about to begin. Maybe I need a

begin. Maybe I need a whistle. All right, everyone. Let's wrap

whistle. All right, everyone. Let's wrap

it up. Let's wrap it up and get to your seats please.

[Music] I can see people walking slowly, ending conversations. And sponsors, if you have

conversations. And sponsors, if you have someone talking to you, just kick him away. Just wrap it up. You can close the

away. Just wrap it up. You can close the deal later on. All

on. All right, get back in your seat.

[Music] All right, people. I would love to

people. I would love to know who here did not check Slack in the last two hours. Anyone not check Slack?

I need to give a shout out. Some people

over here. All of you check Slack. I need to be more of a conference

Slack. I need to be more of a conference bully. Okay.

bully. Okay.

All right. I know where I can sort of hear the conversation is dying down a little bit. I need the teacher tactics.

little bit. I need the teacher tactics.

What's the teacher tactic? Round of

applause.

Yes.

All right. I'm going to follow that cue. If

right. I'm going to follow that cue. If

you can hear my voice, clap once. If you

can hear my voice, clap twice. Shout out.

twice. Shout out.

We're going to kick off the next couple of sessions. It's already been a great

of sessions. It's already been a great day. It's going to be an even better one

day. It's going to be an even better one because we're going to learn a little bit about how to take research out of this home that it inhabits which tend to

be a product sort of home. And so I'm really pleased to bring to the stage Abby who's going to talk about solar powered research today. Please welcome

Abby.

[Applause] Thank you. Uh, great vibes here today. Thank

you. Uh, great vibes here today. Thank

you to the DJs. Yeah. Um, how's

everybody doing?

Yes. Yes. That's awesome. That's

awesome. So, my name is Abby Hoy Balin and it's great to be here with you. A

couple things about me. Uh, I love dance cardio. So, I definitely see a theme

cardio. So, I definitely see a theme here around dance. Uh, and, uh, I also enjoy watching chiropractor videos.

I know. It's a weird It's weird. It's

the ASMR for me. Anyway, okay. So, now

you know a bit about me. Um, I'm the director of user research at a company called Later. And it's a influencer

called Later. And it's a influencer management tool that connects um influencers and brands together to power social campaigns. Some people might call

social campaigns. Some people might call it the dating equivalent um of matching people uh on a social campaign level.

Okay. And then it also helps businesses manage and schedule social media. So I'm

here today to talk to you about solarp powered research. And what does that

powered research. And what does that mean? One of our biggest growth

mean? One of our biggest growth opportunities is providing research to help an entire organization. And today I get to share with you an approach that's worked for me and should hopefully work

for you too. Everyone here is familiar with solar powered, right? And

basically, the closer to the sun you are, the more power that you get. And

I'm going to use this metaphor that this that the sun represents money. And you

are the plants. Okay? And just like how plants, leaves, flowers, trees, they all grow with the sun. You're also going to grow and expand your surface area and coverage. And then all this to say that

coverage. And then all this to say that the closer to the money that you get, the more success you'll have to craft a practice that will drive revenue. And

perhaps some of you might be thinking, uh, I might feel some reservations about getting close to the money, but I want to encourage you. It's okay. Just keep

an open mind. I think it's a great way to evolve the practice.

Uh so today we're going to dig into the how to get closer what this looks like and I'll show you that when it happens research can become an unbelievable partner to the organization and at this very moment we

are actually being proactively accorded to join projects across the company. So

I'm going to show you some of the trends that we've been seeing for our team.

Okay, so this is where I get to put the money where my mouse is. Here's a graph of our projects for the past year in 2024 and the first quarter of 2025. And

in blue are our product projects. In

green are the projects that we're doing through our revenue teams. And by revenue teams, I'm talking about sales, um, customer success, um, and we have an internal agency as well for managed

services. And then pink is marketing.

services. And then pink is marketing.

Um what's interesting about this graph is that we're seeing that downward trend uh for product projects and this could be tied to a particular stage that the product is at and that's not necessarily

a bad thing. It shows a maturing project uh product and a decreasing need for very UX heavy work and a decrease in product requests is also a reflection of a maturing relationship that we have

that research has with product right and we're focusing a lot more on understanding things at a strategic level that forms future road mapaps in terms of the work that we're doing for

the revenue teams in the green so we saw some really big quarterly shifts in 2024 in Q1 and Q2 we started doing a handful full of work that was targeting sales,

deals, and renewals. And that created this huge appetite in um in Q3. So you

see that jump from 18% to 57% about quadrupled the number of requests there. And so that was really

requests there. And so that was really clear that there was this business need, a really strong business need to enhance our sales conversations with these

bespoke insights that research was providing. Um and then for marketing,

providing. Um and then for marketing, there's also been a steady flow of um projects that still increases this year.

And I want to say that while servicing other teams is not a magic bullet for job security, but it does make you more resilient to change. And we all know that the scale of change keeps on

growing and it keeps on changing. Change

changes. So let me paint a pathway for you to have more resilience. Okay. Um let me give you a

resilience. Okay. Um let me give you a bit of background to set the stage first and then walk through some of the factors that led to our growth. Okay. So here's a snapshot of

growth. Okay. So here's a snapshot of the journey we took and how we diversified over time. 2018 uh I joined as the first researcher and research was born. Um and then uh it was very UX

born. Um and then uh it was very UX focused. I was doing all the usability

focused. I was doing all the usability testing. I was running the the beta the

testing. I was running the the beta the betas and briefly even doing UX writing.

Um eventually that UX work evolved into product strategy research and then throughout this time and including today the social media scheduling tool uh of our product was based on a productled

growth strategy and what really changed for our team was in 2022 and later was um acquired and at that point there was

a shift from being a productled uh growth uh company to a salesled growth company and then that was because of the incorporation of an enterprise

management influencer management tool.

And so then we also had to shift. Oh,

now we're not looking at SMBs anymore.

Now we're looking at enterprise customers. So there was a lot of change

customers. So there was a lot of change for us. There was this expansion of

for us. There was this expansion of scope and strategy. It brought in a lot of new people, a lot of new departments into the mix. And one of the first areas outside of product that we worked on um

was to experiment with uh innovation and next frontier initiatives.

Two years after the acquisition was when we started doing uh work for the revenue teams which I showed you in the previous slide and then that takes us to today.

So now we're doing um product strategy research uh market research and uh research for revenue generating initiatives. Okay. So in terms of how

initiatives. Okay. So in terms of how we're structured as a team, um we operate on using a mixture of an embedded service model where each researcher supports multiple product features and then also an agency model

where each researcher will support other areas based on demand. And as much as I love product, I knew that diversification was key to growing

research. Shout out to Zoe. Um, just

research. Shout out to Zoe. Um, just

like how you'd run a business or your investments, diversifying would provide you with multiple income streams and then prevent you from putting all of your eggs into one

basket. Uh, I did want to give a quick

basket. Uh, I did want to give a quick nod to where I first got hooked onto the phrase getting close to the money and it was in 2023. It was at a learners event and what I got out of that was really an

extension of was what I was already thinking around diversification. Um, so

when Roy said, "You got to get close to the money," it was like an unlock for me and it's really been my mantra ever since. Okay, so we've covered my journey

since. Okay, so we've covered my journey with growing research um along with which areas in the company can benefit and the importance of diversification not and not putting all of your eggs into one basket. So let me paint a path

for you to get there too. I'm going to cover four different points. Okay, first and most important

points. Okay, first and most important points. Find teams with external visible

points. Find teams with external visible output because these teams sorry with these teams proprietary data will be your differentiator. And so what do I

your differentiator. And so what do I mean about external visible output? I'm

talking about teams that have external audiences that consume their content.

And so for us this was marketing and sales. Their external audiences are the

sales. Their external audiences are the consumers, the brands, the influencers.

And why is this important? These teams

all value differentiation.

It sets a company apart from the competition and this differentiation will come from that unique bespoke research that you provide. So you can look into different areas of the company see how you can enhance and then provide

that value and then ask will your output differentiate you from the competitors.

So again this is the most important point. If you walk away from anything

point. If you walk away from anything else from this presentation please remember that um I am going to dig into more. So in sales and marketing there's

more. So in sales and marketing there's something known as a conversion funnel.

It illustrates the steps that one goes through from being aware of a product or service all the way to purchasing it.

And at a high level, this is a journey and these are the stages. Start off at the top of the funnel with awareness and consideration. And these stages are

consideration. And these stages are focused on attracting and informing. So

think about it when you hear about a product for the first time. What made

you aware of it? What made you consider it? And then while the bottom of the

it? And then while the bottom of the funnel we have conversion and loyalty which is focused on converting to a purchase and keeping them engaged. So I want to first focus on

engaged. So I want to first focus on awareness and consideration because this is where our research for the marketing function sits. Marketing is in the business of

sits. Marketing is in the business of thought leadership and actually so are all of you. One of the best ways to achieve

you. One of the best ways to achieve that is through proprietary data. The

reports that we partner with marketing is unique to our pool of creators. And

here are some of the examples. We have

the creator rates report and that talks about creator compensation. Done reports

on creator mental health. And more

recently, um, we worked with marketing to supplement a blog article that talks about what the Tik Tok ban meant to creators. Spoiler alert, creators were

creators. Spoiler alert, creators were not impressed.

um they're very concerned about the potential loss of income. They're also

uh concerned about the loss of creativity and community too. So this

type of research is going to have a direct tie into lead generation. And

that's one thing you can measure. Other

metrics might include report downloads, SEO, keyword rankings, qualified page qualified leads, page views, and more.

And these are metrics that your marketing team will be collecting anyway. And so it can also be shared

anyway. And so it can also be shared with you. When I reflect back a couple

with you. When I reflect back a couple years into 2023, we'd maybe have two maybe three reports with marketing on an asneeded

basis. Now, we've mapped out key

basis. Now, we've mapped out key tentpole projects for marketing for the entire year. Ultimately, the impact of

entire year. Ultimately, the impact of our research resulted in a shift from them seeing us as an experimental partner to now being a key always on partner.

Okay, so now going back to the funnel, I want to focus on the bottom area of converting to sale. And this is where our research for the revenue teams come into play. Why is this important? Again,

into play. Why is this important? Again,

it goes back to proprietary data and how that's your differentiator. When we can provide

differentiator. When we can provide insights that teams can't get anywhere else, then it separates us from the competitors and we show up as knowledge leaders in the space, which gives our customers and prospects more confidence

in choosing us. So here are some examples of the work that we've done for the revenue teams. Okay. A primary method that we

teams. Okay. A primary method that we use are surveys to help with the sales and revenue teams when they close deals.

So including research insights into these sales pitches show our investment into their business. So we'll typically ask questions around um creator consumer

motivations, their online habits, what they value, and this output ultimately

feeds into a larger sales um narrative um and why we're best positioned to service them. And let me tell you that

service them. And let me tell you that when you get to see your research into an actual sales pitch deck, it's really really powerful because it gets you so close to the money. so close to that

deal. For metrics, you might want to

deal. For metrics, you might want to measure ACV, which is annual contract value, or TCV, total contract value. We

look at both. And you might also, and this is not something I included in this slide, but you also might want to track the impact of your insights and where that data was used. So often if you do

something for one industry, it's going to be applicable to many businesses within that same industry. So it might be applicable to multiple sales deals, sales pitches. So to give you an idea of

sales pitches. So to give you an idea of that impact, when looking back of Q3 and Q4 of last year, we worked on about 10 revenue projects that included new

sales, renewals, the purchase of additional services, and that had collective TCV of over $1.9 million. And that's a

great data point to share with senior leaders when you're talking about the impact of research.

So now research is really part of that uh winning formula for strategy and sales when it comes to putting together uh sales pitches. I want to show you another example and this slide was taken

straight out of a deck um that we did on behalf of the strategy team and it was for a major US airline back in August

2024. That's when we did the

2024. That's when we did the research and um the structure of this pretty much followed the previous slide to a tea. So that outline and we

surveyed creators on the topic of assigned seating policy of that major US airline. And although we didn't actually

airline. And although we didn't actually win any additional sales from that piece of work at the time, the research was reused in multiple areas of sales and customer success. And it also informed

customer success. And it also informed how to better work with creators because now we could understand their travel patterns. And finally, it was actually

patterns. And finally, it was actually the first push of many that eventually opened the doors to a deal that was worth over a million

dollars. That deal was closed.

dollars. That deal was closed.

Um, it was great because it brought us back into this nice loop where um, the very first campaign that that US airline

wanted to do was on that topic of assigned seating. So, from start to

assigned seating. So, from start to finish, the deal closed 7 months after our initial research, but

it's a great example of investing in the payoff. Okay. The previous step

payoff. Okay. The previous step naturally leads into this one. Flexing

your skill set and to stretch and lean into quantitative methods initially initially at least. So why

quantitative over qual and notice I said initially uh I want to make it clear there's always a place for qual always always. But when you're having these

always. But when you're having these conversations when these teams have these conversations and they communicate with the customers and the prospects having datadriven conversations can be

super convincing. You know, when I say

super convincing. You know, when I say that uh nano influencers and micro influencers have the highest engagement to um a company who has no idea how to

do their campaign strategy and they'll be like, okay, I'm going to put my money there, right? And then if they're

there, right? And then if they're wondering, how often should I be working with creators? Should I just have them

with creators? Should I just have them once or have an ongoing relationship?

Well, I can say that 70% of brands want to have an ongoing relationship with their creators and then also 74% of creators want to have a long-term relationship

with these brands. So altogether when we have that

brands. So altogether when we have that that data shows that we can leverage and inform and we can use that to help the customer make key decisions especially around whether or not to spend their

money with us. So, I would use quantitative to shape the story as a base here and then layer on the qualitative which can come in the form of quotes. We've typically included

of quotes. We've typically included open-ended questions within our surveys.

Um, and after you've done at least one of these projects, then you can layer on more discovery research. I also know that qualitative can work first and then

quant. Um, sometimes the sales teams

quant. Um, sometimes the sales teams might have these really burning questions that can be best answered through qualitative means. And that's

great. I'm just sharing with you what's worked for us. Okay. Other ways you're going to

us. Okay. Other ways you're going to have to flex. You're going to have to be nimble. No surprise here. We are all

nimble. No surprise here. We are all being asked to do more with less. Um I

get it. So we're a team of five and we've definitely had to stretch and see how to combine resources and requests especially across so many areas. I also

want to say that indicative is okay and specifically more for sales because often research is one input out of many, right? and just know that your work is

right? and just know that your work is there to support a larger narrative. You can also complement

narrative. You can also complement proprietary with secondary data.

Remember to practice that. So, we've

been digging a lot more into perplexity and then using thirdparty databases like Statista. The third is to forge key

Statista. The third is to forge key partnerships. Deeper research is powered

partnerships. Deeper research is powered by data competitive intelligence and senior leadership support. So here, who here has already worked with the data

teams? Yeah, of course. Yeah, that's

teams? Yeah, of course. Yeah, that's

right. That's awesome. I'm actually not going to touch on that at all. Um, and

it's because I know that as a collective research practice, we know to do this, but I would like to tell you a couple of stories around competitive intelligence and getting support from senior

leadership.

So, at Later, we're actually really fortunate to have one competitive intelligence manager who sits in the product marketing function. Hey, Matt

Powell. Uh, CI is as much a part of our research team as our researchers because not only does CI look at competitors, the team also looks at the broader

landscape. So our teams meet on a

landscape. So our teams meet on a regular basis to talk about what's in our pipeline and then CI would often supplement that with um their

competitive insights. So an example of

competitive insights. So an example of how we've worked together is through voice of the market presentations um where we'd both present to our executives around what's happening at different

horizons and that would include current product feature investments um at and then at the six to 12 month horizon and then beyond. So we would provide these

then beyond. So we would provide these research findings to reinforce the horizons and then CI would immediately follow by providing the latest outlook on what our competitors are doing and

what could be happening tomorrow. And

what's great about this is that it gives a really nice holistic view about what's happening in the market. Okay, so that's kind of a segue

market. Okay, so that's kind of a segue into the senior leadership support. So

I'm I'm also fortunate to have one of the co-founders as my boss. Hey Sean.

Um and uh who can help navigate and advise which areas to expand to as well as market our services init internally so that we have that initial backlog. I

want to tell you a story about um earlier this year in February the company held this internal leadership summit and we were in the same room as the seauite folks and there were

probably at least 50 other senior leaders in the room.

um unprompted research was endorsed by the chief strategy officer, the chief customer officer, the chief revenue officer, even the chief people officer.

And it was this whoa really powerful moment. It was just like oh my gosh. And

moment. It was just like oh my gosh. And

it was like this cornerstone moment. Um

and it was we were largely recognized due to the work we were doing on the revenue teams actually. But it would have been pretty hard to get there if I didn't get the guidance. So which begs

the question, you know, what if you don't have inroads with the SLT yet? I

would say first start doing the research anyway. Do the work, right? And just

anyway. Do the work, right? And just

reach out to those experts and IC's on the ground to build that set of case studies. And maybe it's about talking to

studies. And maybe it's about talking to the salesperson and maybe it's talking to the um marketing content writer, right? And then once you have a set of

right? And then once you have a set of case studies then you run those examples with their leaders. So if you can make their leader more knowledgeable about

the customer it empowers them and in turns make them makes them a champion of your work. Okay. So fourth and last point is

work. Okay. So fourth and last point is to be proactive and preemptive. What if

you anticipated a need before it was known? What if you gave something

known? What if you gave something valuable that was never asked for? To

grow the way that we have, we've had to be really proactive about uh about what we do, but also um know the gaps that we could fill in. So, I'm

going to walk you through a key example for us. And for that key example, it's

for us. And for that key example, it's this consumer research work. I know it's a really busy slide. I just got really excited and I just slapped on a bunch of stuff and I was like, "Oh my gosh." Um

but there actually was no stakeholder ask for this but it easily became one of the most influential pieces of work that our team had done. We had seen a gap internally and there was little

documented primary research on how consumers interacted with the creator economy. So we went ahead to try to

economy. So we went ahead to try to figure that out ourselves. And um the purpose was to uncover how social media impacts the relationship between the consumer and the brand. And the insights

range from understanding how social media influences purchase behavior and trust all the way to attitudes towards the use of generative AI. So since

conducting this early last year, consumer research has become a part of the company's lexicon with the data being cited on our marketing blog references in sales pitch decks and customer and business reviews as well as

being used for internal education purposes. So it's now a tentpole piece

purposes. So it's now a tentpole piece of work that is recurring on an annual basis.

Okay, so let's get you going solar. Remember that proprietary data is

solar. Remember that proprietary data is going to be the key differentiator for teams that create external output. Flex your quantitative skill

output. Flex your quantitative skill set. Forge key partnerships with

set. Forge key partnerships with competitive intelligence, data, and senior leadership. And be proactive and

senior leadership. And be proactive and preemptive. So if you can follow these

preemptive. So if you can follow these tactics, you're going to make inroads to being closer to the money within the company and be in a position to drive profit.

But Abby, what about the clouds? You

know, I'm going to thank my buddy Brian Keiel. He's a fellow researcher for

Keiel. He's a fellow researcher for asking me this question. Um, so dealing with clouds, you know, this can come in the form of self-doubt. Maybe you're getting

self-doubt. Maybe you're getting criticism from other areas saying, "Hey, research does not belong here." And I want to say that as most clouds do in

real life, it's going to dissipate and evaporate after time because your work is going to show for itself. It's going

to show that you've added value because I didn't get here overnight. Um, from

the point of trying something outside of product to where we are today has taken me three years and it's required a significant navigation, but it has all been possible.

So, how much closer can we get to the money? Because we're actually not done

money? Because we're actually not done yet. And what's closer than having

yet. And what's closer than having research become an actual line item in the balance sheet. So, today we're actually working with our sales and strategy teams on the possibility of

selling that same proprietary research that we've been providing internally now externally to prospects. And it's a proof of concept,

prospects. And it's a proof of concept, but is successful. It's going to be the most direct contribution to revenue we've ever been a part

of. So, I'm going to end with a call of

of. So, I'm going to end with a call of action for you and I want you to start a conversation with a team that has visible output. Then choose one

visible output. Then choose one project. Doesn't have to be big. Our

project. Doesn't have to be big. Our

very first project was actually more of an afterthought and it was because another team had asked us a few questions and we happened to be running a survey at the same time and we're like, "Oh yeah, we'll we'll add those

questions in." Little did we know that

questions in." Little did we know that it was going to be the beginning of a very very fruitful relationship. And so I want to say just

relationship. And so I want to say just choose one more one simple project to start and then once you've done that just sit back and then see what magic can happen. Thank

can happen. Thank you. That was wonderful. We're going to

you. That was wonderful. We're going to grab a seat here and uh do some questions. I'll start off with one of my

questions. I'll start off with one of my own and then we'll we'll probably have a one from from the uh the crew here before we we move to the next session.

I'm interested to hear whether you're you had push back from your team and what type of push back you had because I think many of us who do research I know we talked about being close to the money

those years ago and it's maybe not something that folks are accustomed to or even want to be doing. Um did you get any push back from the team u and and what was that? How do you deal with

that? Yeah, I mean

that? Yeah, I mean uh hey team uh they they were great. I

really gotta say that they were fantastic. Um I think that they

fantastic. Um I think that they understood that there was a need for change but at the same time there was a lot of concern about can we do this?

What is sustainable? And I think that's been the ongoing question for us like what is sustainable? How much is too much? Where do we have to push back?

much? Where do we have to push back?

Because um for us the goal really was to elevate research to get it closer to the money to make that really visible that line that tie in that relationship. And

so that meant that oh are we going to have to drop something else right that could feel

important right? And so it does take um

important right? And so it does take um a bit of a balancing act on on trying to prioritize which ones do you know will

actually make it to the finish line when it comes to the money and what won't.

But yes, I I I think sustainability is going to be one of those key challenges for teams. We have time for at least one question.

If it's a short, maybe two. So feel free to put up your hand. There's one one right over there.

By the way, if you're in the chat, we didn't have any questions yet. Maybe for

the next session, feel free to ask questions. We'd love to get some

questions. We'd love to get some questions from online. Uh hello. Hi. Uh

my name is Ibrahim. And one line that you said uh that really resonated with me was um you know, giving data or information that they can't find anywhere else. Um I work on an

anywhere else. Um I work on an adupported platform or product and so I have a sales team, but I find a lot of times they're going to the market research team. uh is there, you know, a

research team. uh is there, you know, a hard line or a blurry line or what does that line look like that UXR can give that a market research team can't um

that makes us unique and valuable? Yeah,

I think that's a really good question.

for us, we actually don't have a market research team. And then so

research team. And then so um so we've aligned ourselves really close to content marketing. And I kind of wonder, you know, what other what

other pools of data that you have access to that the market research team doesn't, right? And I also think that it

doesn't, right? And I also think that it could be something that you ideate and brainstorm with the marketing team on that would that they can release things

that aren't necessarily so closely tied to marketing, but maybe closer tied to product, right? Wouldn't it be

product, right? Wouldn't it be interesting to release something or have a thought leadership piece out that has to do with how people interact with your

product, right? And we talk so much

product, right? And we talk so much about ease of use. I think that can really resonate with people as well.

That's great. We have time for one more run over here.

Oh, there's um our volunteer gun. I'll run my mic to you. Let's go.

gun. I'll run my mic to you. Let's go.

Hello. Um my name is Chen and I have a question about like uh So there's a project like that actually lasts seven months and then one thing I noticed is like when a research last that long it's

actually hard to track the success metric because there's so many teams involved. So like how do you actually

involved. So like how do you actually pull it off? Yeah. So once we've done the piece of research so that actually did not last seven the research did not last seven months. It just took seven

months to close the deal. Right. So

typically that piece of research took um you know a couple weeks to fulfill and then once once that's done we track

things on a spreadsheet right and then when we try to when we track impact once we know that the deal has been closed then

that's updated in the spreadsheet and go that impact what was the TCV of that deal bam so that's what we do so a lot A lot

of it is lagging. So for when you work with the revenue teams, a lot of those indicators are going to be lagging ones.

It'll happen way after. So you just actually have to be really diligent in following up and um and tracking that impact.

It's incredible to hear the journey your team has taken. I want to thank Abby for that great talk. Thank you so much.

All right, we we just keep them coming. Keep them

coming. And I'll remind you as your conference bully, Slack, Teams, let's keep it to the side. Well, a picture is

worth a thousand words, but researchers, we like words, right? It

doesn't have to be that way. Today,

Vicki is going to help us visualize new methods. Let's welcome Vicki to the

methods. Let's welcome Vicki to the stage.

Hello, San Francisco. When I say re, you say search. Re search. Reese. This is my

say search. Re search. Reese. This is my Coachella, so I'm going to treat it like it.

Um, I'll expect my check in the mail, Alec. But anyway, I'm here to talk to

Alec. But anyway, I'm here to talk to you to hopefully try to find that core of humanity and what we love about it that motivated a lot of us to become researchers in the first place.

So before I really dig into it, just want to quickly say hi, my name is Vicky Zeamer. I'm a lead researcher at

Zeamer. I'm a lead researcher at Salesforce. I focus on a AI monetization

Salesforce. I focus on a AI monetization and I've been working in UX research focused on AI for about nine years now.

So I love AI. I've really been an early adopter of trying to bring it into my research practice. And through that

research practice. And through that process, it's given me a lot of thoughts. Um, and so before I share some

thoughts. Um, and so before I share some of those thoughts, I want to ask you all to think about your own experiences. Why

did you become a user researcher? Was it that you love talking

researcher? Was it that you love talking to people? Was it it was the most human

to people? Was it it was the most human feeling part of tech? Um, was it that you're natural empath? What what about it? I want you to hold on to that reason

it? I want you to hold on to that reason for being that got you into this this career. So speaking for myself, I just

career. So speaking for myself, I just love connecting with people, but most of all, I love building things for people with people. Whether that was becoming

with people. Whether that was becoming an architect, an industrial designer or graphic designer, I always knew I wanted to make something with people for people. So it's been so great to see how

people. So it's been so great to see how AI has been able to help us in our research process, do a lot of things in our craft faster. As I mentioned, I've been an early adopter of some of these

me these tools and honestly it saved me months on some research projects and I'm super super grateful.

But also on the other side of the pendulum, once I got my prompt so good, I kind of felt like a copy and paste robot at a certain point and I was like,

whoa, things are happening so quickly and is this actually the career I want after all? If I'm just going to be

after all? If I'm just going to be organizing and being air traffic control for prompts and agents to execute this research, I really want to stay connected to people as we make

everything else faster. Um, so I really my call to action for you all through this talk is to think about how do we bring our users closer to us given the

speed that we're given with AI. Let's

offload all the tedious stuff and use that time to then get closer and even more stronger connections with our users. So get creative

users. So get creative here. This isn't a life or death matter

here. This isn't a life or death matter we're up against, but it's up to us to decide what the future of this industry looks like. The future is what we make

looks like. The future is what we make of it. And we do have a chance to

of it. And we do have a chance to significantly shape how we continue to tell these stories of our customers, how we continue to engage them, that customer experience we're able to bring

into our studies. So, let's be purposeful when we're engaging with people.

It's our chance to reclaim our purpose, to champion the raw power of human creativity, and to facilitate our users to express themselves in new profound ways that maybe when we were doing some

of that tedious stuff became an afterthought. So, let's think about how

afterthought. So, let's think about how we can use creative methods to unlock irreplaceable sparks that every single person we talk to has. And we want to think about creating insights that don't

just inform, but really ignite everything else around us.

So, a lot of us in our jobs, we spend countless hours listening to these really beautiful, rich stories from people. I've worked in B2B for a while,

people. I've worked in B2B for a while, and even just hearing about the frustration people have with API calls.

You can still see how much of somebody's time that they're awake is spent working. And by making their jobs

working. And by making their jobs easier, we can make their lives better, too. So, no matter your context, we're

too. So, no matter your context, we're thinking about the dynamic experiences of humans in this world. So sometimes it feels like we're trying to capture these unique nuance stories and we end up

reducing them to flat words and icons.

I've been guilty of a lot of iconography. Um but taking that cartoon

iconography. Um but taking that cartoon image and being like people are sad when they have to ask for help sometimes is shortch changing not just our users but also ourselves and the creativity we

have as people in this field. So, I want to ask you to challenge the status quo of how we're doing research, how we're interacting with people so we can empower our users to show us how they're

feeling instead of just telling us. So, with that in mind, there's

us. So, with that in mind, there's something I call user-led visualizations. And these are visual

visualizations. And these are visual depictions of processes, mental models, or other situations that are created by our participants, but facilitated by us as

researchers. So, this kind of research

researchers. So, this kind of research isn't just impactful. I found that it it's really essential for business growth. It's able to help show us

growth. It's able to help show us problems that maybe we didn't expect to find um and maybe would have gone unsaid during a conversation because people, I don't know if you know this, they like

to tell you what they think you want to hear. And sometimes when you ask them to

hear. And sometimes when you ask them to actually show you something or write down a process, there's little bits of details that might have not come up in that conversation. Um, it also unlocks

that conversation. Um, it also unlocks interesting opportunities for innovation by, like I said, finding these points that maybe somebody didn't see as important. It also, what I found is

important. It also, what I found is through some of these visual tools, it gives us a language to see how our customers are thinking about these things. And that's really helpful not

things. And that's really helpful not only for product development but for marketing, for sales, for just your business space to understand better

really what's that point of view. Finally, I don't know about you,

view. Finally, I don't know about you, but by like the 15th time I'm moderating a session with the same discussion guide, it gets a little boring

sometimes. Um, and so really thinking

sometimes. Um, and so really thinking creatively about how do we use that time together. So it's way more fun for me as

together. So it's way more fun for me as a researcher, but even more importantly fun for our participants to be involved too. I always want our participants to

too. I always want our participants to leave a session in a better mood than when they came out of and depending on the topic, they could be really hard.

Um, so today I'm going to be giving you two immediately applicable methods.

There are two of my favorite ones and I'll be showing you step-by-step processes for how you can actually execute studies like this. Uh, my goal is that if you have a study running next week that you could quickly pivot and

maybe try one of these or maybe even think about your own way to have users be more visual as part of your research process. Why I think you should really

process. Why I think you should really do this?

Um, it's really about trying to move beyond text summaries and trying to get more from their point of view. With user created collages, it's

view. With user created collages, it's so much easier for them to say how they felt in a situation via graphic depictions, memes sometimes than is than

it could be for them to say. And mind

maps also really give us a blueprint to think about how our users brain works.

So these approaches can capture emotional resonance and a context in a way that conversations sometimes can't.

Um, and it's important to remember that our job is not just to transcribe and summarize what happened in these interview sessions. It's more to be

interview sessions. It's more to be interpreters and storytellers. Let AI do all the transcription and the summarization. It's good at

summarization. It's good at that. So, before I move on, kind of

that. So, before I move on, kind of talking about co-creating in sessions, have any of you with a show of hands, have any of you used some user creation

methods? Cool. Awesome. My kind of

methods? Cool. Awesome. My kind of people. You want to shout out some of

people. You want to shout out some of them what you've done?

Oh, emoji journey maps. Love it. Anyone

else? Participatory design.

Relationship pipeline.

Cool. Having users draw it how they visualize. Exactly. So, this is kind of

visualize. Exactly. So, this is kind of what we're talking about here. So, why

are tools like collages, mind maps, and all the examples that y'all have brought up really valuable, sometimes more so than just a a conversation? Well, it's

really good to provoking deeper conversation. Creating a visualization

conversation. Creating a visualization together isn't about getting out of it and having this nice picture. It's

really a catalyst for a richer dialogue.

You can ask, "Hey, why did you choose this image?" Or, "I noticed you paused

this image?" Or, "I noticed you paused when I asked you this to do this. Tell

me more." are you can see they're thinking in real time as they're executing something or after you can use it as a reflection tool. It also

facilitates vulnerability. Sometimes

depending on what you're talking about, they can be really sensitive topics and just putting into words how you felt in something can be really scary, especially to a stranger that you haven't met before who just added

something to your calendar and you opened a Zoom link. So creating

something visual, like I said, showing a meme or some other cartoons or even emojis can be a good way into those conversations. And finally, it's a good

conversations. And finally, it's a good way to amplify users voices because it provides a direct tangible output into a user's perspective. We're seeing the

user's perspective. We're seeing the interpretations, the connections in their own visual language. This adds a layer to our studies that we often don't get. So when should you really use these

get. So when should you really use these methods? I like to think of it in the

methods? I like to think of it in the middle of these three circles that I'm not going to go into too much, but generative research when you're trying to figure out what you don't know, attitudinal research when you're trying

to find those feelings, those nuances, those attitudes, and then qual research when you're really looking for those rich stories and the why behind. It's

not great for when you need to do a market sizing experience. It's not good for when you need that quantitative giant sample. Although you can do some

giant sample. Although you can do some of this unmodderated at scale, but talk to me later about that. It's a lot but possible. Um, and if you have a super

possible. Um, and if you have a super tight deadline, maybe think about what else is in your arsenal. So, let's dig into it. We'll start with collage. And

into it. We'll start with collage. And

I'm sure a lot of you have done collage either for fun or in school, but let's think about what this would look like in a user research context. So, let's take the stock image of somebody holding up a

key. What do we unlock? So from our

key. What do we unlock? So from our users, this collage gives a tool for our users to really think about those feelings that they have. And one of my

favorite psychological concepts is embodied cognition. And that's the idea

embodied cognition. And that's the idea that we're able to communicate how we're feeling as it relates to how we experience the physical world. Think

about saying, "My heart feels so full or I feel so down." That's really kind of an example of sometimes it's easy for us to externalize our feelings rather than this nebulous feelings that live inside

of us. So collage gives a tool for our

of us. So collage gives a tool for our users to feel to show how they're feeling based on images, memes, text from the world around them. So they can

capture those inputs and give us this unmatched clarity and understanding. What do you get out of

understanding. What do you get out of doing something like collage? Well,

besides a lot of ahas, uh, it's a crystal clear artifact. It's a map of their brain. It helps you better

their brain. It helps you better prioritize as a business. It also helps you figure out your strategy a little stronger. What are those opportunities,

stronger. What are those opportunities, as Brad was saying, and they're really impactful as well. So, when you start about actually

well. So, when you start about actually planning a study for collage, there's a few things that I would suggest you do.

one, you really want to outline what are your learning goals around the study. We

do that for everything, but in this specifically, think about is there an event or an experience, a moment in time, or like I said, a singular experience you're looking to explore. If

you just say, "Hey, in general, your whole life, how did that feel?" That's

going to be a lot harder to make a collage than, "Hey, tell me about uh the first time you had to uh what's something really scary? Shout it out. I

don't know. Yeah. The first time you spoke on a stage, what did it feel like?

Then you can go back into that moment and access. Okay, I'm thinking about

and access. Okay, I'm thinking about that specific moment. So, zero in on a specific event you're looking to explore. Thank you. Um, then you're

explore. Thank you. Um, then you're going to want to imagine what kind of images might somebody want to use to describe this.

So, are they going to want concrete objects to show what this is like? So,

do you want to have physical objects, places, actions, emojis, or something more subjective? Are they going to want

more subjective? Are they going to want to be showing feelings, um, positive, negative, actions in that way? I guess

actions is concrete, but once you figure out, hey, what is somebody might want to use for this, you'll want to think about what tools do we want to give them to create these collages? So, these are three different ways I like to think

about it. Do I want to create pre-made

about it. Do I want to create pre-made assets here? So, sometimes with this,

assets here? So, sometimes with this, I'll create a Google deck and I'll have one slide that's just like 20 different objects and then I'll ask them to curate

based on that their own slide. So, copy

and paste as many as they want. Um, or

you could think of it as like paper dolls. You give them a lot of the

dolls. You give them a lot of the different outfits and they can put them together. You can also have something a

together. You can also have something a little more open-ended where you give them a Figma board or a Google slide.

again and you say go to Google images, find whatever images you want and copy and paste them on here. You can also do something physical too and this is great if you're moderating a workshop in person or something. Have a ton of stuff

on the table and let people cut it out and put it together themselves. The most important thing

themselves. The most important thing once you've figured out all these aspects is that you really want to pressure test this. So have one or two people on your team follow your prompts and see what they come up with throughout both of these methods. What I

really want to encourage you to think about with your your methodology is when I give these instructions, do I get the type of output I'm looking for from my users? Um, this is going to become

users? Um, this is going to become really important with the mind maps as well, but people can interpret things very differently. So, make sure you try

very differently. So, make sure you try it with a few different people so that you get what you're looking for. So,

let's imagine a sample study where I'm trying to figure out how people on my team feel about using AI tools in their work processes.

So to kick it off, I'll give people a blank Google slide and I'll ask them this question and say, "Okay, you have five minutes. Get whatever you want. I'm

five minutes. Get whatever you want. I'm

going to let you be. Turn on some of that music on Figma and we'll come back and chat about it." This is a real life example that I

it." This is a real life example that I got back from one of my participants for this question. And as you could see from

this question. And as you could see from this, like they really went all over the place in the most beautiful way. We have

some old movie references. We have the Raven meme of her chewing nervously. Um

the fingers crossed. We also have some optimistness with this Dancing Queen cartoon too. You can see like the

cartoon too. You can see like the complexity of emoji of emotions here. Um

here's some more examples when I ran this with some other people too. You

could see like you get all sorts of different stuff in the world of their oyster. So once you've done this, you've

oyster. So once you've done this, you've had the people create the collage, now the real conversation happens of, hey, you know, I haven't seen anomorphs in 15

years. Why do you feel like an anomorph?

years. Why do you feel like an anomorph?

Could you tell me more there? Um, or the people holding hands like what does that mean to you? Use this as a starting off point. Um, the monkey one, as you can

point. Um, the monkey one, as you can see, I really relate to all the different outputs being scary bananas when I'm really just looking for a banana sometimes. Um, so just have

banana sometimes. Um, so just have conversations around how these came to be and what it means. Once you have all these, how do

means. Once you have all these, how do you actually synthesize something like this? Because this isn't just like

this? Because this isn't just like post-it notes, let's make affinity mapping and call it a day. I really

suggest that you try to find common patterns and themes in the image choices. Analyze different parts of

choices. Analyze different parts of emotional expression. Um, across all the

emotional expression. Um, across all the images, are there dominant themes that really jump out for you? I found in my experience after 8 to 10 of these maps depending on the complexity of the study

but 8 to 10 is a pretty good number it becomes pretty clear. Now when it comes to sharing back what you found the most important thing is like don't just let these artifacts you're creating dissolve

into the background as you create your report. You can create user profiles

report. You can create user profiles where for each collage you're using that as a basis point to deep dive into a real life person. You can zoom out and

say, "Here's a theme. Here's what it looks like." And then show participant

looks like." And then show participant examples. Or you can even call out from

examples. Or you can even call out from your research. Here are the needs.

your research. Here are the needs.

Here's what we need to change in the product. And here's the scary memes that

product. And here's the scary memes that people are pulling when they're having to do this without what I'm suggesting.

Like, if we don't do this, there's going to be a lot more like Jerry falling onto a pillow exhausted.

So before I move on to the next one, I wanted to do a really bad joke about mind maps.

Um, what do you call a really good research readout based on mind maps? Like you're

showing off your mind maps. It went

really well. Everyone's like, "Oh my gosh." And then somebody stands up and

gosh." And then somebody stands up and says this word to you. Any guesses? That

was mindblowing. And yes, that was a mind-blowing presentation. There's a reason I work in

presentation. There's a reason I work in tech in San Francisco and I'm not a comedian in LA. And this is why. Um, so

again, let me tell you why I love mind maps in the form of a mind map.

Uh, so a mind map is a user-drawn diagram of mental models, tools, systems, logic. It's really helpful for

systems, logic. It's really helpful for us to uncover user cognition and start mapping it out. It also gives us a ground truth of what does something look like. I'll give one of the examples I

like. I'll give one of the examples I used, but it's really helpful for when you're thinking we have no idea what the life of this person looks like, what is involved, what's that ecosystem, what

are those systems. It's a really great tool then. And it also helps your users

tool then. And it also helps your users brainstorm. It gives them a way to

brainstorm. It gives them a way to actually map out and start making the the uh the ambiguous more concrete. So,

it helps with your data collection as well.

Once you've kind of figure out what you're trying to do, I found that mind maps are really helpful for helping to illustrate these three different types of things. I wish I had a better word

of things. I wish I had a better word for them, but if you're trying to figure out processes, what that looks like, you're trying to explore a system of tools or ideas, or if you're trying to

figure out what decisions does somebody make and why, what's that logic process looks like? It's really helpful for for

looks like? It's really helpful for for making the invisible visible here. So, in order to run these, I'm

here. So, in order to run these, I'm sure some of you are familiar with card sorting. We often think about them as

sorting. We often think about them as open card sorts or closed card sorts.

Are we going to give them the stack of the words and have them create their own groups or are we going to give them labels and see how they fit them into our labels? When you're doing mind maps,

our labels? When you're doing mind maps, there's a similar sort of process I've I figured out is you can have it be closed where you can pre-make some assets of

what sort of things you want your users to include in your mind maps and have them then build and assemble their mind maps with that in mind or you can leave it open and have them draw in themselves

and get some really interesting stuff.

Um, there's pros and cons to each and I'm happy to discuss some of them after this talk, but that's really where you want to start. It's for your question which one might be best. No matter what,

I highly, highly suggest that you pressure test your prompts. You're going

to want to think about how do I step someone through the creation of this mind map layer by layer. And let me show you what that means in an example. So

for this sample study, we want to visualize our users digital ecosystems. Like I said, I've worked in B2B for a long time. So, I always think about how

long time. So, I always think about how is somebody using our tools and how does it relate to the other tools in their workflow. So, to begin, you really want

workflow. So, to begin, you really want to warm them up. Um, you can't really just give somebody a piece of paper and say, "Draw me a mind map of your tools."

And walk away. They're going to be like, "Oh, what?" And also be like, "I'm not

"Oh, what?" And also be like, "I'm not creative. I don't know how to do this."

creative. I don't know how to do this."

Um, so it's really helpful to get their juices going by having a conversation before. I like to have an idea in my

before. I like to have an idea in my head of where are we going to go with this mind map and have them start talking about the elements that they might draw on the mind map later. So

this could be hey spend 30 seconds write down every tool you use on a on the daily basis. Just write it down on a

daily basis. Just write it down on a piece of paper or tell me about the most important things in your life. So

they're verbalizing it a little bit before they have to put it on a piece of paper. Um so once then you do give them

paper. Um so once then you do give them that piece of paper. I'll read out a sample flow that I would use to have somebody do a mindm. Okay. So now on

this piece of paper, I want you to start by drawing by writing out the most important tools all over this piece of paper and circle them. So you can imagine this. We see Gmail, Google

imagine this. We see Gmail, Google Drive, all the stuffs. So I say great now circle again the tools you spend the most time on. Um making their edges

extra bold. Now draw lines between the

extra bold. Now draw lines between the tools you wish were more integrated. And

finally, I want you to shade the tools you're really happy with the way they were green and the ones you're not happy with red. We end up getting this map. So

with red. We end up getting this map. So

once this map is complete, you have a really great conversation starter. You

say, "Oof, what makes uh Google and Slack, what makes those really hard for you? What makes Gmail and Salesforce

you? What makes Gmail and Salesforce work better for you? Why should these things be connected?" You can kind of dig in from there. Here's another

example of a real mind map I made with a user participant in like a 15-inute session around their sales process, why they were using certain tools to report what kind of

[Music] information. Um, here's another one. Um,

information. Um, here's another one. Um,

also looking at people's digital tools.

When you go to synthesize this, it can be kind of overwhelming, especially if you're not used to analyzing something like this. But just take a step back and

like this. But just take a step back and know these mind maps will still exist.

So, let's just take the information you need for this study at this pass, and you can always come back and look at more information later. You'll want to be identifying central concepts and key themes you you're seeing come up. Look

at the different connections and hierarchies that emerge by somebody drawing something big versus other things little. All of that stuff's

things little. All of that stuff's intentional information from humans, whether they were consciously thinking about it or not. Compare these aspects across different participants. Are

people doing the same sort of size for this circle on different things? Do

people crossing it out? But my biggest favorite thing to do with these mind maps is if you're working in person, or you could do this digitally, put these in a folder, go get a go get a coffee with somebody, an executive leadership

or one of your stakeholders before you start annotating them, and put them on the table. Say, "What do you see? What

the table. Say, "What do you see? What

sticks out to you?" Um, I met with the CTO of HubSpot when I worked there and showed them this and he was seeing connections that I hadn't even thought about, but that's what he was thinking about and it's really interesting way to

involve your stakeholders in the process as well. So, when you share back, make sure

well. So, when you share back, make sure again you're bringing these to the forefront by bringing in the visuals and the annotations here and you're summarizing what you're seeing across

these maps. So in conclusion, now that

these maps. So in conclusion, now that we've explored the power of collage and mind maps in action, what does it mean for us? Well, these are just two

for us? Well, these are just two examples. You all helped me by providing

examples. You all helped me by providing some other ones. Some other ones I really like using are those timelines and actually writing them out and saying, "What has changed when you

started doing this and digging into it?"

Um, I also really like in prototype walkthroughs, getting really hands-on with it and having them like cross things out, shading things, like pasting new things and co-building that together. So, I encourage you to

together. So, I encourage you to brainstorm other techniques that might be relevant. And let me know what you

be relevant. And let me know what you think of. I'm always trying to think

think of. I'm always trying to think creatively about how do we make better use with our with our people, especially now that we have AI to help us with some of these more peripheral tasks and the tedious ones. Let's let's get back to

tedious ones. Let's let's get back to that creativity of people. So research

is such an iterative process. So some of these might take a while to perfect. So

even if you try this once and it doesn't work how how you thought, take that learning into the next one. So I want to challenge you all to think differently about how we spend time with users. And

like I said, especially as we start offloading some of this AI, what do we want to hold on to and really keep going with? How might we transform the

with? How might we transform the qualitative research to capture experiences beyond words and beyond those conversations? So the answer it's

those conversations? So the answer it's a call to action. I want you to think beyond those words, create visual artifacts and start being more creative with your process and start

experimenting. Let me know. Thank

experimenting. Let me know. Thank

you. Right. Let's grab a seat and grab some questions. That was brilliant. as

some questions. That was brilliant. as

you were speaking, I was just thinking, why am why why haven't I done mind maps with people in such a long time? Sort

of. It's a great provocation. All right,

let's get some questions. Uh would love if you could raise your hand and um we have a live uh stream question. Oh,

fantastic. Uh so Christina asks about how do you best create and organize a system mind map that has many different

modes that achieve the same results?

So, I'm going to assume the question is about how do you help users create something like that? And I would really start as you're creating the methodology

of listing out what do you want to be included in this mind map. And then for each of those objects, think about what would the instruction be for somebody to do this out. So, like I said, think of

this as like layers on the onion or I don't know if any of you have used tracing paper. you know, you do like one

tracing paper. you know, you do like one layer and you put another layer and you can create a full-on image after all these layers. That's how you should be

these layers. That's how you should be thinking about creating those mind maps.

So, think about what are the central topics or ideas that are going to be the anchors of this. Um, in a B2B setting, this is often your data, where is your data stored, what sort of data is going

in and out of it. What's that looking like? in a consumer se sector that could

like? in a consumer se sector that could be more or B toC that could be more you know if you're hiking and this is for hiking boots how do you decide what shoes to wear like why don't we

write down all the different types of shoes you have and then we could think about the decision-m process coming off of that so think about what is that center of the onion that then all the other layers are going to sit on top of

and just take it one bit at a time if you tell somebody create this map with these 25 things in it you're just you're it's not going to go well. It's great

advice. Thanks so much. All right. Um,

we did have a question from the audience here.

Thanks. Um, first of all, great talk. So

fun and imaginative. It really got, you know, the wheels turning in my head about how we can um, implement these things. And I work with accessibility

things. And I work with accessibility clients sometimes that are blind or low vision. So, I was curious if you had any

vision. So, I was curious if you had any ideas about how to make the collage activity more accessible for folks that maybe would um face barriers trying to

use the um images especially. Yeah. So,

I think this could be something that maybe you even talk them through what they might want to include. So,

rather than asking them to be the ones putting it together, but you know, hey, are there items that you use that you find really helpful in this? and just

think about more like what's their experience of the physical world still and kind of create that in with the with your assistance too. I don't have too much experience

too. I don't have too much experience doing that. So I wish I had more but I

doing that. So I wish I had more but I would love to talk to you after and we can sort of figure out what that would look like because I would be really curious to figure it out. That's a great question. I think we have time for one

question. I think we have time for one more if we have another one here.

Thanks. Thanks. Great talk. Uh quick one on these mind maps. Do you use a model or some sort of data science tool to curate them or is it a lot of quality

time reflecting on it? So, I've been using these now for oh my gosh, too many years. I've started with drawing them

years. I've started with drawing them and I've slowly moved to digital as we've started doing more digital research. One thing I just started

research. One thing I just started experimenting with is using um some of the generative AI to help analyze them.

Is that kind of what you're asking? I'm

not having users create them. So far, I haven't found a good tool that represents accurately what users are looking for, and I want them just to be thinking about their processes rather

than trying to fine-tune their prompts in the research session to get what they're looking for. Um, but potentially in the future, that could be a direction to go in. I would say so far, I've not

been too impressed with generative AI's ability to analyze them. M it is good at sort of reading doing the the word recognition and pulling out like hey they mentioned Salesforce this many

times but in terms of looking at the hierarchies and across a lot of different ones I haven't been too happy with the synthesis yet but that's not to say in six months it will still be

there. Great. Thanks.

there. Great. Thanks.

Tremendous. All right. That was Yeah, that got me thinking a lot. So, thank

you so much for for sharing that. I give

round of applause to Vicki for a great session. Thank you. All right.

session. Thank you. All right.

All right. It's lunch time. We are on a break and we have some catered lunch which is going to it's going to be amazing. I know that a lot of thought

amazing. I know that a lot of thought has gone into the catering, but before you all head out, let me invite Dee up because Dee is actually going to talk us

through um what we have on offer. If Dee

is around, is Dee here? All right. I'm not sure if Maggie

here? All right. I'm not sure if Maggie and Alec are around here because they there are some details about what is on offer today catering wise with any for

folks who who have some restrictions.

There's Dee. Dee if you can hear me. She's walking up on stage

me. She's walking up on stage [Music] slowly. All right, just a little pause.

slowly. All right, just a little pause.

Chat amongst yourselves for a few seconds. Hey. Hey. Hey.

seconds. Hey. Hey. Hey.

[Music] [Music] Introducing Listen, the first AI

customer researcher. Listen speaks to

customer researcher. Listen speaks to thousands of people per week and tells you exactly what they want and why.

Here's how it works. I want to find out what marketers think about our website.

With this prompt, Listen creates a discussion guide. Listen then taps into our network

guide. Listen then taps into our network of millions of users to find the perfect people to interview.

Within seconds, listen starts hundreds of conversations in parallel. What do

you think about this site? Looks cool.

What's cool about it? I mean, having an AI that can help you go through in one hour listeners done thousands of interviews and turns them into an executive summary, common

themes, standout quotes, video highlight reels, and even slides.

Iconic brands like Microsoft, Canva, and Chubbies already use Listen to make their most important decisions faster.

We've raised 27 million from Sequoia to make every company customer obsessed.

Comment listen and we'll give you access and $1,000 worth of listen credits.

[Music] Down.

[Music] Nat.

[Music]

Here we go. Heat.

[Music]

Heat. Heat. Heat. N.

Heat. Heat. Heat. N.

[Music] Heat. Heat.

Heat. Heat.

[Music] Heat.

[Music]

Heat.

[Music] Heat. Hey. Hey.

Heat. Hey. Hey.

Hey. Heat. Heat. N.

[Music]

[Music]

[Music] Hey. Hey.

Hey. Hey.

Hey. Hey, hey, hey.

[Music]

Heat.

Heat. Heat. Heat. N.

[Music]

Hey hey hey.

[Music] [Music]

Woo! Hey! Hey!

Woo! Hey! Hey!

[Music]

[Music]

Why? Heat.

Why? Heat.

[Music] [Applause]

Heat. Heat.

Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

[Music]

[Music] Woo!

Woo! Heat. Heat.

[Music]

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey [Music] hey.

[Applause] [Music] [Music]

Yeah yeah [Music]

yeah.

[Music]

Yeah yeah yeah.

[Music] [Music]

Hey hey hey.

[Music]

Yeah. Yeah. Yeah.

Yeah. Yeah. Yeah.

[Music]

Black.

[Music]

Black. Yeah.

Black. Yeah.

[Music] [Music]

[Music] [Music] [Music] [Music]

[Music]

[Music] [Music] [Music] Hey hey hey.

[Music] [Music] Hey. Hey. Hey.

Hey. Hey. Hey.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

[Music]

Woo! Woo!

Woo! Woo!

[Music]

[Music]

Hey hey [Music] hey.

[Music] [Music]

down. Hey

down. Hey

[Music]

Heat. Heat.

Heat. Heat.

N. Heat. Heat. N.

[Music]

Hey, [Music] hey hey.

[Music] Down. Hey.

Down. Hey.

Hey. Heat.

[Music]

Heat. Heat. Heat. N.

Heat. Heat. Heat. N.

[Music] [Music]

[Music]

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey

[Music] hey.

[Applause] [Music] [Music]

Yeah yeah [Music]

yeah.

[Music]

Yeah yeah [Music] yeah. Heat. Heat. N.

yeah. Heat. Heat. N.

[Music]

[Music]

Black. Yeah. Yeah.

Black. Yeah. Yeah.

[Music] Hey,

hey hey.

[Music]

Hey yeah.

[Music]

Yeah. Yeah. Hey, hey, hey.

Yeah. Yeah. Hey, hey, hey.

[Music]

Heat.

[Music] Heat. Heat. Heat. N.

Heat. Heat. Heat. N.

[Music]

Heat. Heat.

Heat. Heat.

[Music] [Music] [Music]

Woo! Woo!

Woo! Woo!

[Music] Heat. Heat.

Heat. Heat.

[Music]

Oh hey.

[Music]

[Music] [Music] I want

to

be I want to let Heat. Heat.

[Music] [Music]

[Music]

Oh yeah.

[Music]

Hey. Hey. Hey.

Hey. Hey. Hey.

[Music]

Hey [Music]

everybody. Heat.

everybody. Heat.

[Music] Heat. Heat. Heat.

Heat. Heat. Heat.

[Music]

[Music]

[Music] Boot up. Black.

Boot up. Black.

[Music]

down. Hey

down. Hey [Music]

Oh yeah.

[Music]

Hey. Hey.

Hey. Hey.

[Music] [Applause] [Music]

Come here.

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Heat. Hey, Heat.

Heat. Hey, Heat.

[Music]

Down [Music]

hey fire.

[Music]

Hey hey hey.

[Music] Hey hey

[Music]

hey. All

hey. All

[Music]

right.

[Music] Hey

[Music] Yeah heat.

[Music] [Music] [Music] [Music] [Music]

Hey, I can't [Music]

[Music] Hey.

Ooh. Hey.

[Music]

[Music] Hey hey hey.

[Music] [Music] [Music]

Hey hey hey.

[Music]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

Heat. Hey, Heat.

Heat. Hey, Heat.

[Music]

Down

hey I'm a

[Music]

Hey hey hey hey hey.

[Music]

down. Hey

down. Hey

[Music] All

[Music] right. Hey

right. Hey [Music] I'll [Music]

be down.

[Music]

[Music] [Applause] [Music]

Yeah. Hey.

Yeah. Hey.

[Music] Hey.

[Music]

Yeah yeah yeah.

[Music]

Yeah. Yeah.

Yeah. Yeah.

[Music]

[Music]

Yeah. Yeah. Yeah.

Yeah. Yeah. Yeah.

[Music]

Hey hey hey.

[Music] [Music]

Yeah. Yeah.

Yeah. Yeah.

[Music]

[Music] Hey hey hey.

[Music] Heat.

[Music] Heat. Heat. Heat. N.

Heat. Heat. Heat. N.

[Music]

Heat. Heat.

Heat. Heat.

[Music] [Music] [Music] Woo!

[Music] Woo! Heat. Heat. N.

Woo! Heat. Heat. N.

[Music]

[Music]

Hey, [Music] hey hey.

[Music] Wow. Heat.

Wow. Heat.

[Music] [Music] Heat.

[Music] Heat.

[Music] Heat.

Heat.

[Music] Heat. Heat.

Heat. Heat.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music] Heat.

[Music]

Heat. Heat. Heat. N.

Heat. Heat. Heat. N.

[Music]

[Music]

[Music]

[Music]

Woo!

[Music]

[Music]

Woo! Woo!

Woo! Woo!

[Music]

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Hey hey

[Music]

Every moment, every idea, every breakthrough starts with a human being.

Research isn't about data. It's about

people. But in reality, researchers spend 80% of their time wrangling data and only 20% uncovering real insights.

In a world flooded with information, how do you find what matters? You start with Research HX, the AI enhanced solution built for researchers. Researchers who

know that behind every number is a real person, a story, a spark. Imagine

harnessing data as it rolls in to make smarter decisions faster. From fieldwork

to final reporting, sample management handled. Data analysis automated.

handled. Data analysis automated.

Insight sharing just one click away. No

more siloed teams and disconnected tools. Research HX brings it all

tools. Research HX brings it all together in one seamless platform.

Because when you listen deeply, you see clearly. You act boldly. Research HX.

clearly. You act boldly. Research HX.

Efficiency intelligence insight all in one.

[Music]

[Music]

[Music]

Hey hey hey.

[Music] You should [Music] [Music]

be very important that the integrity of the raffle.

[Music] Where's the key?

All righty.

Let's employ some teacher tactics again.

If you can hear my voice, clap once. If you can hear my voice clap two

once. If you can hear my voice clap two times. If you can hear my voice clap

times. If you can hear my voice clap once. If you can hear my voice, clap

once. If you can hear my voice, clap twice.

All right, I'm now now firmly in my arsenal of uh of tactics. Well, we are about to get going with an amazing slate of afternoon talks. So, I'm hoping that

this lunch was really helpful for you to take a step back, have some great conversations, maybe do some thinking about what was mentioned in the morning sessions, but we've got some great afternoon ones as well. But before we

get to that, we have a raffle free stuff courtesy of Lookback and Crew.

Um Henrik, these are books. Ooh,

that's pretty good. That's a great book over there. Very biased, but there's a

over there. Very biased, but there's a great library here for anyone who wants to expand their research knowledge and look back. Have very graciously offered

look back. Have very graciously offered to give this to someone and you can chuck it in your bag. If you have to go back, good luck with that with your hand luggage. But if you're local, it's all

luggage. But if you're local, it's all good. Okay. So, I'm going to draw and uh

good. Okay. So, I'm going to draw and uh one person. Is it just one person? One

one person. Is it just one person? One

person. All right. And I'm not Is there any drum roll thingy I actually I You all can do a drum roll for me, please.

Yep. Who's it going to be? Who's it

going to be? Who's it going to? Who's it

going to? Who's it going to be?

Oh, really? Really? Do Do you think they will trust the integrity of I don't know if they will trust the integrity. I

can't think of a more well-deserved.

Yes. And she likes reading. It is true.

All right. Yeah. You say it.

The person who has won this giant stack of books is our fantastic, careless

speaker Zoe.

[Applause] Do you like reading? I really do. Have

you read these already? This is the best. Awesome. Well, here you go. Enjoy.

best. Awesome. Well, here you go. Enjoy.

This is going to go into my 50 books a year, guys. We can all start a book

year, guys. We can all start a book club. All the new people in the field.

club. All the new people in the field.

Now, look how a veteran will get excited about learning more things. That's the

key. Curiosity. Keep going. It's never

done. If you're tired, it's okay. Take a

break and then get back into it. And

then just read more. Read more. More

always. Okay. Thank you. Thank you, Roy.

This was great.

Sweet. All right. Actually, right before we get into our next session, I just want to make a note. John and the amazing team at Maze are and Jane, love

Jane. Um, they have some really cool AI

Jane. Um, they have some really cool AI stuff that they're building into their product. And if you want to check it

product. And if you want to check it out, give them some feedback, they're going to be over in Rad House. Is that

how you say it? Rad House. Rad House. uh

right beside us here. Uh so anytime in between sessions and even after we're done here, head over there or even just head over there if you just want to have a beer with them. They will gladly out

drink you. I I don't know if I'm making

drink you. I I don't know if I'm making promises for them. I shouldn't be, but hey, there you go. It's on It's on Maze. So definitely

go. It's on It's on Maze. So definitely

head out there and um have a good time with them next door. Okay, it's

afternoon time. Who's ready for some afternoon talks? Are you ready?

afternoon talks? Are you ready?

[Applause] All righty.

Well, I have been really intrigued by a lot of the talks this week that have been focused on AI and uh we're going to have a bit more of that with today's session. Molly is going to be talking to

session. Molly is going to be talking to us about how we build to learn. So,

let's give a warm UX welcome to Molly.

[Applause] My name is Molly and I'm really excited to be here to talk about how we can build to

learn. So I'm Molly. This is me. The

learn. So I'm Molly. This is me. The

same photo from the previous slide but made out of felt. And I'm a researcher at Google. I

felt. And I'm a researcher at Google. I

focus on YouTube.

About a decade ago, I was a founding member of a small team at YouTube focused on really exploring new terrain.

Our team is called Visioning and we work with teams across YouTube to figure out what's next, what's around the corner.

And as you might imagine, a couple years ago, the landscape began to shift and a few of my teammates and I saw an opportunity and co-founded YouTube's

Genai Lab. Our mission became how to

Genai Lab. Our mission became how to explore and learn in this new era we are all entering. But before we talk about any

entering. But before we talk about any of that, I want to start by telling you a story. About 10 years ago, I was at at

story. About 10 years ago, I was at at Google and lived here in San Francisco.

I currently live in Chicago, but I joined Google's Sprint Leadership Academy on an inspir inspiration seeking field trip to a majestic Victorian

mansion just right next to Alamo Square.

And we were looking for inspiration and to seek different ways to learn. This

mansion had been transformed into a studio where leaders from companies large and small could come and quickly test their novel ideas. The whole

building was set up to help these leaders shift into a different way of working, shifting away from talking their way to solutions to really making

and learning their way to understand opportunities. I remember our host there

opportunities. I remember our host there was telling us how quickly they would learn. They shared that they would bring

learn. They shared that they would bring in a specialized user every morning and learn from them, but then they would take the afternoons to build taking those learnings and bringing them to

life.

This mansion, these folks who hosted it, they had a seemingly bottomless rolodex of expert makers, people who could prototype anything. You could build a

prototype anything. You could build a product, a service, a digital, a physical tool within a single day. It

was like completely wild and absolutely magical. The companies that visited this

magical. The companies that visited this mansion were empowered to quickly make decisions, exploring and getting feedback ridiculously quickly in order

to identify the best path forward. They

were off to the races. The location was seriously magical and what they were doing was inspiring. Going from insights to actuals in a single day with every

client who came in getting a bespoke experience tailored to their specific needs. But it was all so very limited.

needs. But it was all so very limited.

Very few were able to experience the sense of magic in this mansion. And I

had that feeling of incredible potential yet really frustrating exclusivity stick with me throughout all of these years. But we're 10 years later. Let's

years. But we're 10 years later. Let's

fast forward to today. The really,

really exciting shift is that so much of the magic that came from that mansion is now available at every single one of our fingertips in these tiny little devices

in our pockets and at our desks.

that power to quickly take things in the real world, things that we learn from our users, and turn them into tangible opportunities, and to iterate quickly or to just entirely throw it away and build

something new more quickly than ever.

And ultimately, we're able to do this and move forward in our product decisions with more confidence, creating things that provide real value to real

humans in this very, very real world.

And that is why I'm so excited to be here with all of you today and you online. In the past few years, some of

online. In the past few years, some of the constraints that have defined how we do research have disappeared. We have we together as UX researchers have the opportunity to redefine things. How we

learn and how we build people- centered products in this brand new era that we're all entering. We, everyone in this room, everyone joining us online has a chance right now to do something

different. We have the chance to

different. We have the chance to experiment and we together have the chance to define what UX research will become in this new era. There are so many different things

era. There are so many different things we could talk about today, but I want to start with just three. The first thing is a shift focused on speed and our

speed of learning. How we can move from long cycles to quick loops. The second

is about how we as researchers can be empowered to start making to bring our insights to life with a shift from waiting to making. And the third is how we can make our research stimuli more

relevant to our participants in the room with us. Shifting from generic mocks to

with us. Shifting from generic mocks to personal artifacts. So this will be

personal artifacts. So this will be super quick, but we'll start with the first shift. Really dramatically

first shift. Really dramatically increasing the speed of our learning. We

all know that in today's fast-paced world, the relevance of our insights is often tied back to their timeliness. If

our findings arrive too late, the opportunities are often missed and our influence, our opportunity for influence diminishes.

We've all had that moment where we've experienced a really long comprehensive learning cycle where the time from initial question to actionable insight

can easily take weeks if not months if not quarters maybe a half a year. Um, we

meticulously plan, we develop the research script, we manually, we recruit, we conduct sessions, we manually transcribe our notes, we spend time on thematic analysis, and we craft

detailed reports full of human stories.

These are honestly some of the projects we as researchers are most proud of.

We've gathered deep insight, developed novel frameworks, and created a foundation for our teams and for our companies to be better for our users.

But what happens when our drive for research perfection actually makes us or miss our opportunity for research impact? By the time our polished

impact? By the time our polished findings land, the product teams may have already moved on. The market

conditions may have shifted, making our insights either feel too late or slightly offtarget. Opportunities can be

slightly offtarget. Opportunities can be missed, influence be diminished, and we just couldn't keep pace.

But what if we leveraged this brand new technology to radically accelerate the speed of our learning? About a year ago, I really

learning? About a year ago, I really wanted to test this idea of impossibly quick make learn make cycles. My team

and I gathered to experiment with a new type of insight sprint. In a single week, we explored a really broad theme.

Each day focused on a different lens of that theme that our teams were interested in. We would spend the

interested in. We would spend the mornings learning, learning from lots of different people, learning from our users, learning from analogous experiences and experts. And then we would spend the afternoon building

functional prototypes and research tools based on these learnings. This may look familiar based on the mansion example.

Every day we left with really fresh insights and a functional prototypes that we could continue to learn from. The insights that we learned were

from. The insights that we learned were about people at the people level. They

weren't about our people as users. They

captured enduring human needs, those latent and explicit needs that would make people's lives better that have often remained unressed to this point due to technical limitations of the

past. And the wide array of sources we

past. And the wide array of sources we learned from helped us sorry helped us um define core principles. This was a

while back that still continue to inform our work to this day. And the

prototypes, they were like the coded version of sketching an idea on a sketchbook or writing a notes on a post-it. They were intentionally rough,

post-it. They were intentionally rough, ready, and meant for that moment. They

were tools designed for exploration, curiosity, and learning. They were

designed to spark new thinking and catalyze new questions. They were by definition not MVPs or like really rough research flows. and they didn't look

research flows. and they didn't look anything like our product. And that was by design. They freed us up from the

by design. They freed us up from the constraints of the day-to-day and allowed us to truly explore novel capabilities without getting bogged down in today's limitations. They simply made

our insights real enough to feel. Together, we began to see the

feel. Together, we began to see the possibilities in this new era at the intersection of enduring human needs and novel Genai capabilities.

The impact of this approach was transformative. Throughout the week, our

transformative. Throughout the week, our team was encouraged to see with fresh eyes, listen with open ears. The whole

mindset of the week was about opportunity exploration and curiosity. Instead of one long cycle, we

curiosity. Instead of one long cycle, we were living in these absurdly quick make learn make learn make learn make loops. Five loops, five cycles in

make loops. Five loops, five cycles in five days. The insights gathered from

five days. The insights gathered from each morning fueled creation.

and ideation for the next day and helped inspire research questions. We explored

so many possibilities in such a really brief time. We had no idea what was

brief time. We had no idea what was going to be important at that moment.

But with new users coming in every day, we had the opportunity to continue learning, iterating, and exploring at a really quick pace. The evidence from

these cycles and learning loops made the potential of Genai tangible. Armed with

these rapid insights, we had a much deeper understanding of our users and the teams that we were able to work with were able to move forward with even more confidence. So, how do you bring this

confidence. So, how do you bring this into your life? The first thing is to make space. I've learned from many of

make space. I've learned from many of you as we've been talking today that it's tough to do this in your day-to-day life. This method took a week and the

life. This method took a week and the insights from this week continue to inform our work today. It requires a shift in mindset to be able to shift from what we learned earlier today from

that tactical mindset to a strategic mindset. Not looking at the next cycle,

mindset. Not looking at the next cycle, not looking at the next quarter, but looking and imagining what might be. It

requires us to make space to get inspired, to learn not only from our users, but from analogous experiences, experts, and extremes to see how an

unexpected perspective might inspire our work. And most importantly, it requires

work. And most importantly, it requires all of us as researchers to play, to get hands-on. And that brings me to our next

hands-on. And that brings me to our next shift, which is the shift of waiting to making. Um, it's really cool to see how

making. Um, it's really cool to see how this technology can empower us as researchers to bring our insights to life. One thing that was really cool

life. One thing that was really cool about the insight sprint was not just the speed, but it was who was doing the making and how that shifted our

dynamic. We've all been there. That

dynamic. We've all been there. That

moment, this is me. I usually have my hair in a side braid.

um that moment we uncover an unexpected insight, that aha where we've met with a user, they've noted something and we've discovered a novel opportunity or chance

that we can actually make the world make this user's life better, help them overcome a hurdle, we are sure to pull that insight into a report with a bunch of other user stories and learnings and

recommendations and hand it off to our team and then we wait.

We want to see those insights translate into something real to be produced in the real world. Our initial excitement can turn to frustration as insights

beyond like a doodle mean a handoff, another layer of translation and another game of telephone. We are dependent as researchers on so many others to bring

our insights to life. But in that insight sprint, that wasn't the case.

After our morning sessions of learning, it wasn't just the designers or the engineers who were bringing these ideas to life. The whole team was empowered to

to life. The whole team was empowered to bring their insights to life almost immediately with the help of Genai. A designer on the team played the

Genai. A designer on the team played the role of engineer immediately vibe coding new experiences into existence and experimenting with novel interaction

patterns. A product manager on the team,

patterns. A product manager on the team, he was really developing the output of a potential experience. he was thinking of

potential experience. he was thinking of essentially visualizing his PRD. And another researcher on the team

PRD. And another researcher on the team used audio generators to turn those insights into a concept album, bringing the insights to life in an entirely new medium. It wasn't just a creative

medium. It wasn't just a creative exercise, but it was a way to test the boundaries of this new technology. Notice what's happening

technology. Notice what's happening here. Independent of their traditional

here. Independent of their traditional making role on these teams, everybody in the room was empowered to translate their insights into something tangible.

Almost immediately, they were empowered to try these new tools and they were empowered to experiment. The wild thing is it wasn't just about our product teams. It's something that we're seeing

extend to our users as well. The work we do is in really deep partnership with our users. I remember reconnecting with

our users. I remember reconnecting with a creator months after we had worked with them. They had participated in a

with them. They had participated in a really extended code design session with us and they showed us how they were bringing this waiting to making mindset into their day-to-day life as

well. They were about to launch a brand

well. They were about to launch a brand new EP and had a specific fantastical hairstyle they w they wanted to have featured on the cover of this album.

They were not able to find any relevant images to share with their stylist. So

they generated images instead. Their

stylist was able to see their vision clearly and tangibly. And now that vision that she had had in her head is now actually available on the cover of

her brand new album. This this is the essence of waiting to making that anybody independent of their background can take insights, can take ideas, can

take their vision of what the future might hold and bring it to life.

I often think about making on this spectrum. On one end, you have the deep

spectrum. On one end, you have the deep technical expertise of an engineer and on the other hand, you have me, somebody who has never coded a thing in her

entire life, but might be able to doodle and conceptualize some ideas. With Genai, it's not about

ideas. With Genai, it's not about everybody needing to be a coder or a designer. Instead, it's about all of us

designer. Instead, it's about all of us lowering that barrier to entry for tangible exploration. About giving us,

tangible exploration. About giving us, you, me, other researchers the ability to quickly sketch out an idea, visualize a concept, or even build a simple

functional exploration. And this is all possible

exploration. And this is all possible because we're moving away from an era where humans had to act like computers and towards an era where computers are beginning to act a little bit more human. We can collaborate them. We can

human. We can collaborate them. We can

chat with them. We can talk with them in a human language the way that we might with our colleagues, our friends, and they can help us bring once impossible ideas to life. And I really have to be

clear here, there is no way that we are aiming to replace any of our crossunctional colleagues. Their craft,

crossunctional colleagues. Their craft, their depth of skill, their ability to create polished, scalable, robust products is irreplaceable and

essential. The goal here is very

essential. The goal here is very different. It's about broadening our

different. It's about broadening our toolkit as researchers and helping us become better teammates. It's about enabling us to

teammates. It's about enabling us to make insights real enough to feel, bringing them to life in ways that we hadn't been able to before. Reducing

that game of telephone. It's about us being able to show rather than tell. And

it's about us being able to collaborate in actuals. really creating a better

in actuals. really creating a better starting point for conversation and functional collaboration and we as researchers are gonna get it very wrong. The artifacts

that we are creating are going to be intentionally rough and ready and by any expert's perspective very very wrong.

But their wrongness can actually be by design. Their purpose is not to be the

design. Their purpose is not to be the final solution, but to be a catalyst for inquiry, for feedback, a tool for shared

understanding, and a spark for deeper inquiry. You can quickly bring your

inquiry. You can quickly bring your ideas to life. You can ensure that your insights

life. You can ensure that your insights are not just heard, but seen, felt, and iterated on. You can bring your crossf

iterated on. You can bring your crossf functional partners not just a list of findings but tangible explorations that serve as a really rich foundation for

their work fostering more effective collaboration. So I have a request for

collaboration. So I have a request for you in the next week try to spend 30 minutes with a genai tool to create create a quick tangible representation

of a recent insight you learned. Maybe

it's a chatbot maybe it's an image generator. Maybe it's a music generator

generator. Maybe it's a music generator and you want to make a concept album.

Just try to make something and don't aim for perfection. Just aim for

for perfection. Just aim for something. Our next shift is being able

something. Our next shift is being able to shift with AI the the stimuli that we create with our users. Being able to shift from generic mocks to personal

artifacts. We've often had to create a

artifacts. We've often had to create a one set of stimuli for all of our participants. This has historically been

participants. This has historically been a practical necessity, often due to resources, and we have to force ourselves as researchers to prioritize

the average user. But our

participants, they're not averages.

They're they're unique. They live in a complex world that is increasingly personalized to them. Why shouldn't our research stimuli

them. Why shouldn't our research stimuli reflect that? as

reflect that? as well. Research tools in our era of Genai

well. Research tools in our era of Genai can become more flexible and personal than ever before. And this can happen at every stage of the product development cycle. Not just when you're getting

cycle. Not just when you're getting ready to launch that people are seeing themselves in the product, but really in the very first day of trying to explore a new idea. Niche can become the new

normal with individuals, not averages, dictating importance.

There are a ton of different ways you can personalize research stimuli. Of

course, you can ask your participants specific questions in a screener and then you could take time to handcraft and handcreate specific stimuli for that

session leading to feedback that feels for that particular user, not for some hypothetical user. But today, we can go

hypothetical user. But today, we can go so much further by prototyping with real data. More and more Genai tools allow

data. More and more Genai tools allow you to inform their brain or their knowledge base with specific information. You can upload context and

information. You can upload context and resources, save detailed instructions, and transform these tools into specialized collaborators. Imagine a researcher

collaborators. Imagine a researcher exploring a new concept where personalization is key. Instead of

generic placeholders with the right informed consent and privacy, they can actually have participants contribute relevant data to that session. They can

take things like text that they have written. They could take things like

written. They could take things like images they've created, lists that they manage, even data that they have downloaded specifically for this particular session and share it with the

researcher. Whatever is pertinent to the

researcher. Whatever is pertinent to the question being explored. This

information can then be imported into AI prototypes in a really lightweight way.

You can learn with something that is truly personal and bespoke to that user.

Suddenly, participants aren't responding to hypothetical data. They're seeing

elements of their own world reflected back to them, making the experience, however conceptual, however early in the process, feel like home.

The shift from abstract concepts to personally grounded experience can unlock rich insights, helping us expose hidden hurdles, capture emergent opportunities, and helping us as

researchers learn from reality, revealing insights that generic examples simply wouldn't. It allows us to get so

simply wouldn't. It allows us to get so much far beyond that surface level feedback and tap into much deeper, more authentic user responses.

So, one more little experiment for you to try in the next week. As we know, a lot of these tools can be informed by your personal data. So, I want you to try to create a Gen AI that knows you.

Upload maybe something like your LinkedIn profile, some research things that you're interested in, maybe an article that you've written, and put that into an AI tool and then compare it

with a generic model and see how the output is different. See then as you experience that how you might incorporate that type of research, that

type of learning and that type of technology of building more personalized AI into your own experience. And if you want a bigger deep dive into how to do this specifically, make sure you stick

around for the very very last talk of the day. There will be very least

the day. There will be very least specific details on how to do this. So it's been a fun day. We've been

this. So it's been a fun day. We've been

on a journey together learning from how we can increase the speed of our research through quicker learning cycles going from long cycles to quick loops.

How we can empower researchers to get hands-on and make their insights tangible moving from waiting to making.

And how we can increase the relevance of our research stimuli by moving from generic mocks to personal artifacts. We've seen how many diff of

artifacts. We've seen how many diff of the different constraints that have defined our practice are dissolving. We

have the unprecedented unprecedented opportunity not just to observe the future but to actively shape it.

Remember that Victorian mansion? That

place where rapid bespoke user- centered learnings happened at an unimaginable pace. Where ideas became real in a snap

pace. Where ideas became real in a snap due to an endless rolodex of makers and where insights were so spectacularly tailored. For so long, that type of

tailored. For so long, that type of power felt exclusive and often out of reach. But the exciting truth is that so

reach. But the exciting truth is that so much of the magic made possible at that mansion, the opportunity to iterate quickly, to be able to learn more

quickly, is no longer limited to the privileged few. With Gen AI, much of

privileged few. With Gen AI, much of that incredible capacity is now available at our fingertips in the tools that we use every day. And this isn't just about quicker

day. And this isn't just about quicker tools or faster processes. It's about

our ability to amplify our core strengths, our ability to have deep empathy for our users, our ability to

uncover human needs, and our ability to continuously advocate for our users.

Genai can help us do all of this more effectively, enabling us to provide clearer, faster, more tangible evidence to help our organizations make smarter, more people- centered decisions with

greater confidence. Genai can help us as

confidence. Genai can help us as researchers reduce risk, increase clarity and confidence, and ultimately build more people- centered products

that provide real value.

To me, the future of UX research is incredibly exciting. It is more

incredibly exciting. It is more hands-on, more iterative, and will allow us to be more impactful than ever. So,

with that in mind, I want to give you one final challenge, and that is to play with AI, to give it a try, to experiment, to explore, and to play. Do

not let those inevitable stumbles that will come along as you try get in the way of you experimenting and trying to bring this work into your life in new

ways. So together, let's build to learn.

ways. So together, let's build to learn.

Hey friends, it's awesome. Let's grab a seat. We have a few minutes for some

seat. We have a few minutes for some questions. I love the uh the challenge

questions. I love the uh the challenge to just do something this week. You

don't have to wait. Just go and do it.

So, um, yeah, that's great. All right,

let's take some questions. I know we have a mic about, so I think we have a hand up in the middle over here. Testing. Sweet. Hi, my name's

here. Testing. Sweet. Hi, my name's Akquil. Great presentation. Um, I was

Akquil. Great presentation. Um, I was just curious if you run into stakeholders that are like really reluctive for their research teams to

use AI as a fellow optimist. I I think it's really important that we share the optimism, but uh how would you suggest

to people who might have uh stakeholders or management that are really reluctant to use or to tell their teams to use genai in some format? what would be your

response to um how to approach it with a more optimistic approach? Yeah, I think a big thing for me that's come across as I've been a founding member of the

visioning team and co-founded the Genai lab and tried a bunch of new stuff is none of it is like loved right away or even understood right away, but we do a lot

of quick little experiments and get a bunch of quick wins that suddenly spiral into us being really appreciated. So if

you can find those opportunities to incorporate Genai into your research, into your research process, into your research methods, that you can really start to build those little snowballs

that can get people hyped. And I think there's a big thing that like a lot of times people might be hesitant to use this technology when they've only heard of it but haven't really played with it.

Once folks get hands-on with it, it's pretty evident that they could see the value for them specifically in their day-to-day life and they start to also

see the edges of it where they as humans are necessary and critical to the development of this work as well. So,

it's really, as we've heard a bunch of times this week, a partnership of people and machines jamming together to make awesome stuff happen, but to start small to get those quick wins and then to

watch it snowball.

Great. All right. Any more questions? I

think we have time for one more one over here. Hi

here. Hi I can but I don't know if the YouTube can. Can you hear me now? Yes. Okay,

can. Can you hear me now? Yes. Okay,

perfect. Well, thank you so much for this talk. It was so great. Um, and I

this talk. It was so great. Um, and I was curious for the insight sprint. I've

run like more traditional design sprints and I found scope can have such an impact on the success of it. And I was curious if there's any considerations for an inside sprint, whether it's scope

or something else, to really make sure it's like successful and and yields something that your team can work with.

Yeah, I think it's been really helpful.

In this one in particular, we had a really specific theme that was not targeted at any one particular team, but that we had buyin from many teams across

the company that this would be helpful.

And the insights were we always say like we're learning in ways that are like 10 sizes too large for any particular team but that any team could find some value

in like if that shirt t-shirt jacket is 10 sizes too long like some element of it could provide value for a lot of specific teams across the company. So a

lot of it is on us as researchers then to translate the insights, the stories, the prototypes that we've built to land within specific teams and for them to

really see themselves within it. So it

is very by design a giant massive scope on a big gnarly question, but then it's the translation and the communication back to teams where we can make sure that they see themselves in it and can

act on those insights as well.

Um, one quick one for me before we wrap up. How's the folks you work with? Oh,

up. How's the folks you work with? Oh,

um, especially researchers. What's

openness been to to do this? Like what

are those? Yeah. What's openness been like to just embrace uh using Genai? I

think for the specific researchers I work with, you met some of them on Tuesday of this week. You might have met Renado who leads the insight lab at YouTube. like the specific orbit that

YouTube. like the specific orbit that we're in, we are encouraged to experiment, to play, to test the boundaries and to explore. There are

people and definitely I've seen this in working with students and others as well that the hesitancy is real. So to find those ways, to find those opportunities

where folks can try something specific, maybe try something little again, to try a little snowball where they can start to see themselves in these tools, to see

the real human value. Again, it's not about replacing us or replacing any of our teammates, but things might evolve in terms of our role in the research and to be able to play with that and see it

and how the technology can make us and our products more human. I love it.

Thank you for the talk. Thank you for the encouragement. Let's give a warm

the encouragement. Let's give a warm round of applause to Molly.

Thank you.

Okay.

So, going to switch gears. I'm going to talk about relations. Relationships. Uh, anyone

relations. Relationships. Uh, anyone

anyone have a lovehate relationship with PMs? A couple of hands come up. All

PMs? A couple of hands come up. All

right. Well, after this talk, that's going to be sorted. It's only going to it's only going to be love. So, let's

invite Zach to the stage. is going to talk about the five types of PMs every researcher needs to know. So, let's give warm welcome to Zach.

I don't know about that. We'll we'll do our best. Um, hi, I'm Zach. I lead the

our best. Um, hi, I'm Zach. I lead the research team at Door Dash. Uh, I also previously worked for quite a while at Netflix. So, you'll see examples from

Netflix. So, you'll see examples from both Door Dash and Netflix throughout the presentation. I called this the five

the presentation. I called this the five types of PMs that you'll interact with and how to influence them. But PM is interchangeable really here for any kind

of cross functional partner that you run into that you might maybe be a little doubtful about working with research or maybe push back a little bit about working with research and how to

navigate that relationship which I'm not particularly good at. I just want to throw that out there. I learned a lot from a lot of other people which brings me to my next slide. Uh, shout out to all of the people that helped contribute

to the various experiments that I'll be showing today at the top. And also a quick caveat, there's a ton of great PMs. I hope they're watching this. The

others I hope are not necessarily watching this. Um, and next year I'll do

watching this. Um, and next year I'll do a talk on the quirks and difficulties and failings of working with people like me to make it even, but that's not the point of this talk. I roundly

acknowledge the difficulty of working as a PM in tech. Ultimately, the buck stops stops with them. If we come to them with an idea that doesn't really pan out,

we're our job is not on the line, but as a PM, if they continuously have a difficult time moving metrics, then their job is on the line. So, I

acknowledge that challenge and this is hopefully going to help you learn how to work more better with them. Okay. Wrote

a Medium article a couple years ago.

It's about impact. Uh this is a definition I pulled from it which is a significant improvement to the business or product experience that's a direct result of a unique contribution that you have made. That's our goal as

have made. That's our goal as researchers. Ultimately that helps us

researchers. Ultimately that helps us keep our job, helps us get promoted, all that kind of stuff helps us feel good, helps us make an impact. Uh and we need to combine that goal with what our

target is. Our target as researchers is

target is. Our target as researchers is to find the needs that satisfy both goals of our audiences that we work with and the goals of the business. There are

moments where you will do things that are better for audiences that don't necessarily move business metrics. I'll

give one example later in the talk.

There are moments when businesses tend to overindex on earnings and money and forget the needs of audiences. But as

researchers, our unique contribution can be to find that middle point. So, if

we're all aligned on that, let's talk about how to get there. Oftentimes, it's

very difficult. Um, for you Lord of the Rings nerds out there, I put this picture and then I was like, you know, it doesn't actually really work because who's blocking who in this picture, right? If you've ever seen the movie.

right? If you've ever seen the movie.

So, you can interpret who's the PM and who's the researcher. I'll leave it up to you. All right. Uh, okay. Here's the

to you. All right. Uh, okay. Here's the

five types of PM. You got to have like this slide in every presentation. We'll

go through each of them. Number one.

Okay. So, if you have kids, I have kids.

love kids. Uh, this book is great. If

you've never read it to your kids, you should definitely buy it. Uh, the key here is that you actually have to have a good idea. Okay, let's start there. If

good idea. Okay, let's start there. If

it's not a good idea, none of this actually works. But if you have a good

actually works. But if you have a good idea, then you can move on. All right,

number one, the love language PM. If

you've heard of the love languages, it's essentially the way people like to give gifts and the the way people like to receive gifts, and it's kind of a relationship thing. I'll give you a

relationship thing. I'll give you a quote, direct quote from a dasher who went to try to deliver a gift to somebody in rural Texas. And as they

were pulling up, they were surrounded on this road by a bunch of men with guns on ATVs and threatened and to that that they would be shot if they didn't get off the property. And they said, "I'm a

dasher and it's a gift for Door Dash."

They said, "I don't want this. Get off

my property." Right? I use this example to illustrate the kinds of things that people might experience while engaging with your product that might be uncommon or things you might personally have not

had experience with. We brought this among many other examples to our PM leadership and uh we were able to get a bunch of empathy built for these

uncommon situations which resulted in the launch of these safe dash tools. Um,

these are things like if you're out dashing and you feel unsafe, you can immediately call things like ADT or you can do location sharing, things along those lines. That is like the golden

those lines. That is like the golden path for qualitative research. That has

happened quite rarely in my career. That

is one example where it has happened quite often. What uh what happens is you

quite often. What uh what happens is you walk in and there's going to be some kind of quote and it will generally look something like this. Well, you only

talked to five people or you only surveyed 300 people. That'll come from the this will come from the analytics people who like do millions of data points. This will come from the PMs

points. This will come from the PMs generally speaking. Okay. So, when

generally speaking. Okay. So, when

you're confronted with this quote, which I'm sure you've all heard before, there's lots of ways to get around it.

And the key is to speak to the love language of whatever it is that the PM is has. Okay, here's an example where

is has. Okay, here's an example where the golden path didn't work. the head of Netflix mobile design came to me and said, "Hey, I'm a mobile designer and I have to open the Netflix app every

single day, multiple times a day in order to do my job and I work here and I can't find it on the screen." That this is probably an issue. My guess is that the design is suboptimal. So, we went to

the chief product officer. We went to the VP of product and said, "Hey, what do you think?" Of course, who wants to change their mobile app design and their brand based on like feedback from one

person? So we went and we did a test

person? So we went and we did a test where we created you'll see at the top of the screen those are like target apps and we give them a little bit of a brief

exposure time to the target app and then we will display a personalized prototype with apps that are actually on their phone and we say find that target app as quickly as

possible. And then we just looked

possible. And then we just looked objectively at how difficult it was to find the various apps. And uh the white Netflix app, the old Netflix app from

like I don't know eight years ago was literally one of the worst ones. Snapchat was the best. Shout out

ones. Snapchat was the best. Shout out

Snapchat. Um okay, so that led to an entire new app design, but it wasn't just from our research. There was

actually a big rebranding effort that was going on that we were able to like ride the boat with and we were able to create this new design which is actually a lot easier to find on your phone if you've ever tried to do it. But in this

case, we're speaking with quantitative objective data. We're not appealing

objective data. We're not appealing through empathy. We're dealing with

through empathy. We're dealing with people who are really metrics driven. Um

here's another great example. Uh we

there was a bunch of social media chatter and there was a bunch of conversations from dashers about the quality of our hot bags and whether they kept food hot that was supposed to be hot and cold that was supposed to be

cold. Okay. And so you'll see here's a

cold. Okay. And so you'll see here's a thread from Reddit where someone was comparing it poorly to the GrubHub bag.

And so we knew right what to do in this case. Actually the left side is a bit

case. Actually the left side is a bit cut off. What we did, and this is what I

cut off. What we did, and this is what I this is my number one recommendation here, is to think about many experiments that you can bring to life a metric that

a PM will care about. So in this case, we literally drove to McDonald's with nine different hot bags, bought nine McFlurryries, stuck them in the hot bags, drove back to the office, and stuck a thermometer in them to figure out what the temperature was. Quite

simple. We did the same thing with miso soup to see how hot it remained in the hot bags. And the Door Dash hot bag was

hot bags. And the Door Dash hot bag was not that great. And we then said, "Hey everyone, it's not that great." And

now you can see the new hot bag. Um,

this did not take that long to get impact to be honest with you because we kind of knew who we were speaking with and we geared the metrics toward them.

And now you can see a new quote from Reddit which I pulled last week which I can't read. It says, "That's awesome.

can't read. It says, "That's awesome.

Been using GrubHub's version hoping one day for Door Dash to have one of these."

So that made me feel really good. So,

here's a couple examples of other experiments we've done. One thing we did was we took user uh written drop off instructions at difficult to drop off

locations and then had AI rewrite them, printed them out, handed them to dashers outside of the apartment complex and got a stopwatch out and timed them how long it took them to drop them off. It was

faster with AI instructions. Another

thing we did is we sat in a restaurant and ordered the same meal from Door Dash, Uber Eatats, and GrubHub and waited for all of the delivery people to show up and figured out who had the most

accurate arrival times. And you can see those metrics underneath. They're very

uh PME ccentric metrics. Every single

example I'm giving you up here mattered and has changed something about an algorithm or a product experience at Door Dash. Okay. Number two, the end of

Door Dash. Okay. Number two, the end of 1 PM. Our favorite. I'm sure you know

1 PM. Our favorite. I'm sure you know exactly what this is. Uh first off, this is not actually always a bad thing. And

this is true in research in general. Uh

this is an example from a qualitative uh research project at Netflix where uh we had a signup screen. This is the new signup screen which you can see on on uh

behind me and it said uh enter your credit card number here at the time. And

one participant said I can't sign up then. And we were like what? Why can't

then. And we were like what? Why can't

you sign up? Come on. They were like, I don't have a credit card. And we said, well, you know, you can just put a debit card in. And they're like, oh, I have a

card in. And they're like, oh, I have a debit card. And so all they did, the PM

debit card. And so all they did, the PM was like, okay, let's just add or debit two two words to that sentence. And

revenue went through the roof. Right? So

you think, is N of one necessarily a bad thing? No. Here's a good example. But

thing? No. Here's a good example. But

sometimes it can be. For those of you who work at Uber Eats and Instacart, we talk about bananas way too much uh at Door Dash. Oftentimes, this is about

Door Dash. Oftentimes, this is about ripeness and how many people want. Do

you want order in bunches or singles?

There's a lot of things about bananas.

Anyway, don't get me started. We talked

about this yesterday. Um bananas can be a tactical

yesterday. Um bananas can be a tactical distraction because it's essentially one fruit that stands in for the idea of customization of what you're trying to order, right? Uh are they the most

order, right? Uh are they the most popular thing? Are they the most popular

popular thing? Are they the most popular thing by volume, money, any of that kind of stuff? No, they're not. But they tend

of stuff? No, they're not. But they tend to come up an inordinate amount of time in conversations with senior leadership.

Um, okay. And also, you might have a kind of a mediocre idea as your NF1.

This is an actual example. Uh, if you've got ever gone to Netflix on the web or TV mobile, there's always a way to get to categories, genres, if you will. So,

there was a theory out there that we should personalize the appearance or the location of this sort of category thing, if you will, and it was called category cravers, which everyone always has cute

names for projects, right? This one was called category cravers, which you'll get to see that we made fun of later.

Uh, that said, we didn't necessarily think that this was like the thing we should go after in that moment. It

seemed kind of tactical or maybe it wasn't a great way of testing whether something was a good idea generically or not. So, we had lots of conversations

not. So, we had lots of conversations with our PM in this moment to kind of figure out how to move forward. So, if

you hear phrases like, "Hey, can you validate this idea for me?" They're

coming at that conversation as if the idea is a good one rather than as if the idea is a hypothesis. So, you need to reframe that as well that's a great

hypothesis. We will test that hypothesis

hypothesis. We will test that hypothesis rather than I will go and validate your idea. Another thing that you might hear

idea. Another thing that you might hear is they're often like sitting in one session, right? They'll show up for one

session, right? They'll show up for one session and they'll be like, "Oh man, well, in the session I went to that's a that italics is on purpose. I went to, I

heard the one person say the thing that I thought that I liked before I even went to that session." That's the kind of stuff that you might hear. Okay. So I

was talking to my team about this and the biggest suggestion that I thought was super useful from that I heard from a researcher on my team is that they will plant seeds early and often

throughout research. So they'll do quick

throughout research. So they'll do quick summaries after every single participant in Slack or to the PM. uh they'll do uh debrief sessions at the end of every

single day to make sure that everyone's hearing multiple insights from multiple people and then they're keeping everyone aware as they go. And over time, the insights will take on a life of their

own. And at by the end of the

own. And at by the end of the experiment, whoever the product manager is at that time will sort of adopt the idea that they've been hearing over and

over and over from the researcher as if it was their own idea. and they become the end of one about the good idea and not the end of one about the idea that kind of sucked in the first place. Uh so

uh we call that inception. I would

definitely do it. It really works really well. Um okay, now let's make fun of

well. Um okay, now let's make fun of that name. We made shirts to make fun of

that name. We made shirts to make fun of that name. Uh where who needs cravings

that name. Uh where who needs cravings for chocolate, cigarettes, coffee, carbs, or connection? It's categories we all want. Yeah. So this is this is I

all want. Yeah. So this is this is I have this at home. We all wore it around the office. All right, enough of that.

the office. All right, enough of that.

All right, the next one, the chief product manager officer. Okay, the these are people that overindex to senior leadership. I'm sure you're all quite

leadership. I'm sure you're all quite familiar with them. Okay, here's an example of a couple insights from actual research we did on the merchant portal.

The merchant portal is where merchants go in order to essentially manage their business. They might edit their menu.

business. They might edit their menu.

They might set their store hours. They

might run an ad or a promotion. They

might check their financials, etc. Uh, this was one of the most entertaining uh list of NPS quotes I read. Uh, this is a couple years ago where they complained about doing the menu editor and the

first one's like, "It only took it took me 15 seconds." And the next one's like, "It took me over an hour." And then the third one's like, "It took me a week."

And then the last one's like, "It took me forever." So, I thought that was

me forever." So, I thought that was really funny, that progression. Anyway,

all right. I'm going to show you that it actually does kind of take forever.

Okay, let's see if this works if I press play. Nope, it didn't. I knew it

play. Nope, it didn't. I knew it wouldn't. Okay, can you press play,

wouldn't. Okay, can you press play, please? Okay, this is the portal. Notice

please? Okay, this is the portal. Notice

how they're scrolled about a third of the way down the page and they're going to 86 something. I've asked them to do this. Okay, they're clicking deactivate

this. Okay, they're clicking deactivate now and now it's going to take a little while to load. Do you see how the thing scrolled all the way back to the top of

the page? Oh my god. Right. I know.

the page? Oh my god. Right. I know.

Okay, so you're a merchant and you have a particular menu item that you Has anyone worked in a restaurant before?

Okay, sweet. Shout out. Yeah, I used to work at McDonald's, Hardies, and Friendlys. I probably served you food at

Friendlys. I probably served you food at some point. Uh, so you run out of something

point. Uh, so you run out of something specific. Okay, let's say it's a

specific. Okay, let's say it's a particular kind of cheese. There are a lot of menu items that will have that kind of cheese and you so you'll scroll to the first one. You'll deactivate it.

Then you'll wait for it to load and then it'll scroll you back to the top. Then

you have to scroll back to the next one and you have to deactivate it. Scroll

you back to the top. You get what I'm saying? It's it's a loop and loop and it

saying? It's it's a loop and loop and it goes on forever. And these were legitimate complaints, right? Okay. So

you go to the PM with this. They're

like, "Ah, my road map is slammed.

Sorry, I just got pulled into a fire drill by the CEO." Or, "Thanks." This is my least favorite quote. If anyone ever tells you this, you've really failed as a

researcher. That was really

researcher. That was really interesting. Okay. All right. So, what

interesting. Okay. All right. So, what

we did is we had the senior leadership in the merchant team try to create their own menu on the merchant portal. Okay,

it this is a I'm I'm only going to play a minute of this. This is a 12minute video of him like, oh my god, this is terrible. This is just one like it took

terrible. This is just one like it took forever to load screen. All right, let's play the video.

I want to do a modifier. There's a couple options in

modifier. There's a couple options in here. This is really

here. This is really [Music] confusing. Cup. Let's add an

confusing. Cup. Let's add an [Music] option. Bowl,

option. Bowl, cup. Um, I don't know. Lid,

cup. Um, I don't know. Lid,

whatever. Uh,

dollar$ dollar50.

Okay.

[Music] Rules. Okay. Save

Rules. Okay. Save

changes. One 2 3 4 5 6 7 8 9 10 11.

Okay. 11 seconds to make that change.

All right. 11 seconds to make that change. So the first person at 15

change. So the first person at 15 seconds was the winner. Okay, let's see if it goes to the next slide. I

sometimes it will replay. Oh, okay. So

we've adopted this principle of bringing in senior leadership to do things that are kind of painful that lead to insights. Uh and we will even do it when

insights. Uh and we will even do it when it's early days on something. This is an actual experiment that we ran in the parking lot next to work in San Francisco where we are working with

Burger King to create an AI voice drive-thru experience. Uh, and it's not

drive-thru experience. Uh, and it's not perfect, right? But it's early days. So,

perfect, right? But it's early days. So,

we had actual participants drive up in their car and order stuff from the Burger King menu and we learned a lot about it. And then we had the VP of

about it. And then we had the VP of product do the exact same thing and the VP of the area do the same thing and the VP of design do the exact same thing.

They all came in and they pretended to order on the Burger King screen so that they could become empathetic of what the experience actually was like. And so

going back to the previous example, this is the new merchant menu experience for the it's called the self-s served menu editor. Um, and so we brought a lot of

editor. Um, and so we brought a lot of the stuff to the surface. We created a search at the top in case you wanted to jump to something really quick. You were

able to collapse menus. You're able to change item availability right next to it instantly. And also uh we were able

it instantly. And also uh we were able to reduce load times by uh at least 63%.

This data is a couple of years old. So

you saw that that that that load time took about 11 seconds. We got that down to about 2 to 3 seconds. But it was an an incredible amount of work and we had to call code yellow and we had to focus

on this rather than some of the other ideas that we were working on. Uh but

really what unlocked it was the fact that we had brought in or that the executives got involved and that our insights bubbled up to executives. Now

am I saying that needs to always happen?

No, of course not. But what I am saying is that if you run into PMs that will only listen to them and not to you, get show the person they'll listen to what you need to show them so that the so

that the the request comes down from them. Okay, that's number three. number

them. Okay, that's number three. number

four and number five. These are the two paradox PMs. They basically say the exact opposite thing of one another, and you will have no idea which one it is

until they open their mouths. Okay, so a couple years ago, there was an This is actually 2019, so that'll date this. Uh

there was an SNL sketch where they made fun of how difficult it was fi it was to find things on Netflix and they they created the endless scroll plan where all you did was get to scroll and you

had to pay for it. Do you all remember this? No. See okay. Yeah. So I I mean

this? No. See okay. Yeah. So I I mean the user need is fairly obvious and we knew that it existed forever before this, right? Even since the dawn of it's

this, right? Even since the dawn of it's true still today on quite many streaming services, right? Netflix equals choice.

services, right? Netflix equals choice.

So there's a tedious world of choosing out there. Okay. So we created this idea

out there. Okay. So we created this idea of like hey let's find some way of bringing to the surface something interesting or exciting without them having to think too much and scroll. So

this play something idea was born and you can see some of the initial design executions here. Uh we went to research

executions here. Uh we went to research and showed it to people. Um, then we took it to AB test and in both of those cases it failed pretty spectacularly

because people had this expectation that it was going to get them to uh continue watching even quicker, right? And so we were using the wrong algorithms, the

wrong entry point, the wrong value prop and all of this stuff we told them about in a meeting PMS etc. And I'll never forget this. Uh, what did they say in

forget this. Uh, what did they say in that meeting? And they said, again, you

that meeting? And they said, again, you got to think, it's important to get the uh the punctuation correct on this one.

Just insights. This isn't actionable, right? Okay. You'll see why in a couple

right? Okay. You'll see why in a couple of slides later. Okay. Just insights.

Okay. Uh the the person on the team that I was working with on this at at the time, I I I sent her a text and I said, "Do you remember that meeting?" She's

like, "Yeah, I remember meeting." And I can do you one better. I have a picture of the actual whiteboard that we took in that meeting after we drew stuff all

over the board. This this is the original like vibe coding right here. So

what we did in the room at the time was we recommended a new goal. We

recommended a new entry point and a new icon and we pushed them away from continue watching and more towards content discovery or low barrier to entry content that was more the kinds of

stuff that you might accidentally run into if you were flipping on TV channels. And when that product was

channels. And when that product was launched, that did improve streaming and discovery. And we're we don't get me

discovery. And we're we don't get me wrong, we're all about coming up with cool ideas, pitching new ideas to all sorts of crossf functional partners, but you never know if you walk into that

room if that PM is going to be uh receptive to those ideas or they're going to put up barriers to those ideas.

Which brings me to uh the the new thing which uh this basically take Molly's entire presentation and this is one slide to say exactly what she's saying

which in order for this kind of PM that's saying hey these aren't actionable. We now take all of these new

actionable. We now take all of these new AI tools and we literally just make them actionable. Uh and it's worked really

actionable. Uh and it's worked really well again. You'll see literally more

well again. You'll see literally more banana. This is literally about bananas

banana. This is literally about bananas again here. Okay. Okay. So, this is a

again here. Okay. Okay. So, this is a usable prototype that we can illustrate using new AI tools. These are ones that we've used. Um, you should try them all

we've used. Um, you should try them all out, as many as you want. There are pros and cons to each of them. Uh, but the key here is to bring to life what you're trying to say so that they can it's not doesn't have to be perfect. Exactly like

Molly was saying, it doesn't have to be perfect. It just has to get them pushing

perfect. It just has to get them pushing in the right direction. It has to get their juices flowing. Okay. But what if you walk into the room? Okay, this is why you need that the punctuation to

work. Uh, this is from the PM. It

work. Uh, this is from the PM. It

doesn't have any question marks. They

just say just insights. That's a

command. Just insights. Don't do my job.

Okay, these are people who feel threatened by your work that you are trying to essentially do their job for them and they say like, hey, I'm trained. I am the keeper of all of the

trained. I am the keeper of all of the abilities to do this thing. I see this a lot from the research side as well where researchers will go hey I am the trained

researcher only I am able to do quantitative and qualitative research you have no skills in this area stay out of my realm both of those situations are

completely wrong. Uh but let me show you

completely wrong. Uh but let me show you in this case what we do. What we do is we literally take an explicit recommendation and we turn it into a how

might we statement. This is this is my hot tip. So this is the PM saying your

hot tip. So this is the PM saying your idea is dumb. We should build a new entry point for the context when members are actually fatigued. Like this is you saying I got a great idea and the PM says that's dumb. But if you went in and

said, "How might we build a new entry point for the context when members are actually fatigued?" They'll be like, "That

fatigued?" They'll be like, "That interesting concept. Let me brainstorm

interesting concept. Let me brainstorm with you and come up with some ideas."

This is my hot tip. It's very easy to do. Just change a couple of words and

do. Just change a couple of words and they will be more receptive to your ideas. But you never know who you're

ideas. But you never know who you're going to run into. It will either be like these aren't actionable or it will be don't do don't do my job for me.

Okay, that's it. Here's your summary slide. love language PMs, you've got to

slide. love language PMs, you've got to speak their language. Is it qu? Is it

quant? Is it empathy? Is it data? Is it

behavioral data? Is it all three? Figure

it out. I heard a great quote yesterday from one of the pres presenters who said quite often crossf functional partners will try to convince you of something

using the language that they are most convinced by. Right? So if they come to

convinced by. Right? So if they come to you and say, "Hey, check out this uh you know this metric I pulled from SQL about how often people do this thing that I'm

interested in changing." Then the next time you convince them, go to SQL and find a metric to prove your point as well. Okay. Second, N of one, plant

well. Okay. Second, N of one, plant seeds for inception. The key here is to constantly be feeding in things, right, that you hear right off the bat. have

the Slack thread like the old days where we would have docs that everyone would take notes in constantly be streaming live thoughts as the as your one-on-one conversations go in qualitative research

do summaries constantly make sure they get in front of them and make those people feel like the ideas are theirs ultimately it doesn't matter who gets credit here what matters most is the end

users right okay the third the CPMO make sure the exec does it and make sure the exec understands the pain they quite often don't have time to do this and once they experience the problem they're

going to have the conversation and rep prioritize for you and they will do your job for you and then the last group is the paradox PM one and two try to bring the first one to life so that they can

see it through AI and the second one is all about reframing with how might we statements and that is it thank you all right I Yeah, I have a question. Do

you do you find that these PMs does it hold steady for an individual the whole time you know them or do you find there's like PMS maybe shift from one to

the other? Oh yeah. Um just like any

the other? Oh yeah. Um just like any segmentation, you all do segmentations, right? Um it's a model, right? It

right? Um it's a model, right? It

clusters people together into buckets and some people will be great fits for that segment, some people will be pretty poor fits for that segment. But you have to put everyone into one of those

segments. The same applies here. PMs

segments. The same applies here. PMs

will go in and out of these groups or they might fit in multiple of the groups. The key is to listen for what it

groups. The key is to listen for what it is that they need to hear from you and to figure out the path based on that conversation.

What did you say? Oh, like segmentation.

What do you mean by segments? Yeah, like

a segmentation. like you you you survey a thousand people with 50 questions and you do some cluster analysis on the back end and everyone goes into a bucket kind of deal and then you call them like weekend warrior mom or something. You

know what I mean?

Okay. All right. Let's grab one question from the audience if uh if we have one over there.

Great talk. I'm really looking forward to the researcher flavored one next year. Um keep an eye out for that. My

year. Um keep an eye out for that. My

name is Molly. I'm from Figma. Um, I'm

curious about failure modes. So,

specifically the CPMO. I have a particular person I can imagine. Um, and

I could also imagine that that style wouldn't convince them. They'd be like, "Well, it's easy for me." They're kind of a combo of CPMO and N of one maybe.

Um, I'm just curious about like failure modes. I don't know if there are other

modes. I don't know if there are other like strategies that you've seen used or other ways that you've seen this, you know, fall apart and what you would recommend in that case. Uh, it's a good question. Um, I mean, ultimately they're

question. Um, I mean, ultimately they're every they're all people and they want to be successful. So, one thing that we'll do for people like that is to we'll throw them a bone early on, like

they'll ask us specifically for something and we'll try to be more reactive and do the thing that they want to do to prove that we tend to be worthwhile and that we can help add value for you and we'll try to make them

successful. And then once you sort of

successful. And then once you sort of have your foot in the door, you can start to bleed more into strategic and proactive stuff. I know there was a talk

proactive stuff. I know there was a talk on this this morning, but it's not always like just flip a coin. And a lot of the times it's about just con like relationship techniques just you want to make them

feel valued and successful and then you can start introducing your ideas more successfully. Brilliant. We actually

successfully. Brilliant. We actually

have that was a quick answer so we have time for one more over here.

Hey Zach, thanks so much for that great talk. Um, as mentioned earlier,

talk. Um, as mentioned earlier, oftentimes our teams are have like a pretty low researcher to PM ratio. So on

my team, we have one researcher searcher to like 15 PMs for instance. So what

happens when you get conflicting types of PMS that a single researcher has to sort of work on juggling and appeasing

and trying to sort of triangulate their questions and um any strategies for that? Yeah, that's a great question. Our

that? Yeah, that's a great question. Our

our ratio is like between one researcher to five and one researcher to 15ish depending on who it is. So same boat. Uh

two things. Number one, you get a tiebreaker. And the tiebreaker is quite

tiebreaker. And the tiebreaker is quite often who those 15 PMs report into. Like

the way I structure my team is that I try to have a researcher GPM equivalent.

So the PMs themselves all report up into a GPM and they cover a space. Let's say

it's um I'll just pull one out. Like

merchant success is or one of the things or or um live order experience. that

thing that I was showing you on the menu manager that fits under merchant library experience. There's a GPM above all of

experience. There's a GPM above all of those PMs and we work with them directly on on quarterly basis on prioritization for our path forward even though we get intake from all of the people that

report into them and we also contribute ideas proactively. Number two is and

ideas proactively. Number two is and that what generally works is if you think about what the company's northstar is and then you start moving your way down the chain. What's the company's

northstar? What is the product goals for

northstar? What is the product goals for 2025? What is the areas goals for 2025?

2025? What is the areas goals for 2025?

What is your PM's goals for 2025? And

then you'll see that there are things that will go to multiple layers and that will impact multiple layers. So then the conversation becomes well, we're going

to prioritize this because it it not only satisfies your PM goal, but is aligned with the company metric that we care the most about and you're it's a top three priority for your entire space. And that's why we picked this.

space. And that's why we picked this.

Those two things generally work the best.

Lovely. Well, Zach, thank you so much for that brilliant talk. A round of applause to Zach.

We can we can sit. We can chill. I'll

dismiss everyone. All right, we're going to take our last break of the day before we have two other sessions to round us off. So, take a break, grab some more

off. So, take a break, grab some more coffee, have some more combos. Come

speak to Zach, speak to Molly. We'll see

you at 3.

[Music]

[Applause]

[Music] Enjoy the music.

[Music] What

[Music]

the heat?

[Music] What [Music] the beat up?

[Music] What [Music]

the? What the

the? What the

[Music]

Hey hey [Music]

Hey hey

[Music] hey.

[Music] Yeah,

[Music]

oh yeah.

[Music]

Hey hey hey.

[Music]

Hey hey hey.

[Music]

Yeah. Yeah.

Yeah. Yeah.

[Music]

I'm not

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat.

[Music] [Music]

Come

[Music]

on. Come on.

on. Come on.

[Music]

Hello.

[Music]

Hey woo.

[Music] Heat. Heat.

Heat. Heat.

[Music] [Music]

Hey. Hey. Wow.

Hey. Hey. Wow.

[Music]

Heat. Hey, heat. Hey,

Heat. Hey, heat. Hey,

[Music]

heat. Heat. Heat. N.

heat. Heat. Heat. N.

[Music]

Hey,

[Music]

Hey, hey, hey. Heat. Heat.

[Music] N. Heat. Heat. N.

N. Heat. Heat. N.

[Music] [Music]

[Music]

[Music]

Heat. Heat.

Heat. Heat.

[Music]

Hey,

hey hey.

[Music]

Hey hey hey.

[Music]

[Music] down.

Hey Heat.

[Music] Heat.

[Music] Yeah down.

[Music] [Music]

Hey. Hey. Hey.

Hey. Hey. Hey.

[Music]

[Applause] [Music]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

[Music]

Hey, [Music] hey hey.

[Music] [Music] You want

[Music] me only Let me let me

[Music] [Music] [Music] [Music]

[Music] Hey hey hey hey hey.

[Music] [Music]

Heat. Hey. Hey. Hey.

Heat. Hey. Hey. Hey.

[Music]

[Music] [Music] [Music]

[Music] Heat. Hey, Heat.

Heat. Hey, Heat.

[Music]

Hey. Hey. Ooh.

Hey. Hey. Ooh.

[Music]

[Applause] [Music] [Music] [Music]

me. Yeah.

me. Yeah.

[Music] Yeah.

[Music] Yeah.

[Music] Yeah. Heat. Heat. N.

Yeah. Heat. Heat. N.

[Music]

[Music] Black. Yeah. Yeah.

Black. Yeah. Yeah.

[Music] [Music] Hey hey hey.

[Music] Yeah yeah

[Music] yeah. Hey. Hey.

yeah. Hey. Hey.

[Music]

Hey. Hey, hey, hey.

Hey. Hey, hey, hey.

[Music]

Hey. Hey. Hey.

Hey. Hey. Hey.

[Music]

Oh yeah. Hey. Hey. Hey.

yeah. Hey. Hey. Hey.

[Music] [Music] [Music]

Come [Music] on. Come

on. Come [Music]

on. Heat. Heat. N.

on. Heat. Heat. N.

[Music]

Yeah. Hey. Hey.

Yeah. Hey. Hey.

[Music] Heat.

Heat.

[Music] [Music] Heat. Heat.

Heat. Heat.

[Music]

Heat. Heat.

Heat. Heat.

[Music] [Music]

[Music] Woo!

Yeah! Heat. Heat. N.

[Music]

[Music] [Music] [Music] [Music] Hey hey hey hey.

[Music]

[Music]

[Music] Hey, [Music]

[Music] Heat. Hey, Heat.

Heat. Hey, Heat.

[Music] Ooh. Hey. Hey.

Ooh. Hey. Hey.

[Music]

[Applause] [Music]

[Music] Hey. Hey.

Hey. Hey.

Hey.

[Music] Heat. Heat.

Heat. Heat.

[Applause] Hi, I'm Susie.

Respondents simply spend up to 10 minutes chatting with me. What's your biggest financial goal

me. What's your biggest financial goal this year? Saving up to renovate my

this year? Saving up to renovate my kitchen. Thanks for sharing. Why is that

kitchen. Thanks for sharing. Why is that so important to you? The kitchen is the heart of the home. My family always gathers there. Welcome to a new era in

there. Welcome to a new era in conversational research.

[Music]

Hey hey hey.

[Music]

[Music]

[Music] All right, folks. Gonna urge you to get

back to those seats. We have two more sessions before we wrap up an incredible day. So folks who are outside, please

day. So folks who are outside, please make your way back in. And folks who are around all the amazing sponsor tables, you can start to make your way back to

the seats. We'll start in a minute.

the seats. We'll start in a minute.

[Music] I totally lied. I am coming back on

stage. Um, Maggie sent me up there. Up

stage. Um, Maggie sent me up there. Up

here. Hi, Maggie.

Uh, as Roy said, we are about to wrap up an incredible research week. And I think we have a few more rounds of applause if we can manage it. First and foremost to

our wonderful hosts who happen to be here today. Heather, Zoe, Roy, Sarah, I

here today. Heather, Zoe, Roy, Sarah, I know we're not here, all the wonderful folks. Uh, Jud,

I'm missing one, but I am just too tired to remember what happened on Monday. Um,

another big thank you and round of applause to all of our wonderful speakers throughout the week who taught us so much about research. Please give

it up. One final and quick thank you once

up. One final and quick thank you once again. And I keep stressing this because

again. And I keep stressing this because again, we just literally could not put this on without them. Thank you again for the sponsors who have invested so much in our community and we're so grateful and we will continue to invest

in you and we hope you come back again next year. And if you don't, we'll be

next year. And if you don't, we'll be really sad. Thank you to the

really sad. Thank you to the sponsors. We love you

sponsors. We love you all. We appreciate you all. And with

all. We appreciate you all. And with

that, I think Oh, yeah. The volunteers,

guys. You were killed it, [Music] volunteers. We had some serious

volunteers. We had some serious volunteer security going on. Bouncing

out all the people who shouldn't be here. They're like, "No. Are you a

here. They're like, "No. Are you a researcher? Are you excited about

researcher? Are you excited about research? This place is not for you." If

research? This place is not for you." If

you know something, okay, anyway, you guys are amazing. Um, so with that, if I don't get to see you all again after this, thank you so much for coming. It's

been so wonderful seeing you. have a

wonderful rest of your day, week, and I can't wait to see you all again next year. Please give it up for Roy.

year. Please give it up for Roy.

[Applause] Okay, we got two more

sessions. Two more sessions.

sessions. Two more sessions.

two more opportunities for you to not work, to not focus on the Slack DMs and the emails or the pagers. Anyone still

have a pager? I think pages are coming back, right? If I'm correct. So, if you

back, right? If I'm correct. So, if you have a pager, shout out to you. But hey, let's focus.

you. But hey, let's focus.

So if you have been living the last few years, you know what it's been like when a company changes in a moment. Many of us have experienced

moment. Many of us have experienced that. And I think just the reality of

that. And I think just the reality of business is many of us, maybe hopefully not many of us, a few of us are probably going to experience it in some way in

the coming years. But there's a way to be thinking about this and a way in which we can be thoughtful about what it's like to operate in environments where businesses give us surprises out

of nowhere. So today we're going to

of nowhere. So today we're going to learn about how to pivot and not perish. And Reggie's going to teach us how. So let's give a warm UX

conf welcome to Reggie.

Thank you. Thank you. Oh, I like that. Give it

you. Thank you. Oh, I like that. Give it

up for the DJ. I don't think we've clapped for the DJ. I like that. I like that. Hello

DJ. I like that. I like that. Hello

everyone. I am Reggie and I have one question for everybody in the room. That

includes vendors in the AV. Everybody

and everybody on the live stream, I have one question for you. By show of hands, do you recall? Do

you. By show of hands, do you recall? Do

you remember what it feels like to start a new job? Everybody. Everybody should

be raising their hands. Okay, great. You

know what this feeling feels like?

Especially if it took you a while to find a job, right? Yeah. Those bills do not pay themselves, right? Now, you

remember that feeling that you had when you started that job, the first couple of weeks, you know, you have that sense of excitement. Your new co-workers are

of excitement. Your new co-workers are sending you those welcome to the team shoutouts on Slack. You know, you get those happy emojis on Slack. And

especially if you're in person, as many of us all are, you get those high fives.

People are dapping you up as you go to the desk. You feeling all the love. you

the desk. You feeling all the love. you

feeling all the feels. You're feeling

all this good energy. You know, it's new beginnings, right? You you're developing

beginnings, right? You you're developing new relationships and you feeling like you can capture all the opportunities of that new job. All of the things. What a

great feeling it is to start a new job right now. That's exactly how I felt

right now. That's exactly how I felt three years ago when I started at Zenesk. I was so excited. I was just so

Zenesk. I was so excited. I was just so happy to excited to just leap into this new opportunity for my career. But can

you imagine that after a few weeks you start this job you're getting settled in. You are organizing your 500 Slack

in. You are organizing your 500 Slack channels. You're doing all those things

channels. You're doing all those things just as you are getting excited and gaining momentum. Imagine getting a Slack

momentum. Imagine getting a Slack message that says your company will soon be acquired by private equity and you're going from public to private. Yeah, that

was me, July 2022. And I have to say, the news just rolled into my mind like a raging storm on the seas. I mean, I was like, "What the bleep?" I mean, I felt

all of this, all of these emotions in the first 15 minutes. I'm like, "Holy bleep, you know? I can't believe this bleep. What in the bleep is happening?"

bleep. What in the bleep is happening?"

Bleepy bleep bleep. Now, look, I've been working in

bleep. Now, look, I've been working in tech for a while. You know, I've been around the block a few times. I've seen

a lot of change in companies, but I did not expect this. No, this took me because I just left Twitter for this job. I mean, come on. I already left a

job. I mean, come on. I already left a looming bleep show. I mean, I didn't need any more chaos or problems in my life. I mean, this was really hard. This

life. I mean, this was really hard. This

news shook me, you know? I've been doing this for 20 years. And I'm like, "Oh my god, how did I get myself into this?"

Uh, well, let's keep it real now. Uh

leaving Twitter when I did was a good life decision to say the least, but it just felt like the ground had shifted under my feet. You know, how was I how did I

my feet. You know, how was I how did I get myself caught up into this until another uh a public to private company acquisition situation? I mean, it was

acquisition situation? I mean, it was really unner unnerving. But, you know, I took a moment. I took a moment to breathe and started thinking just like Roy said, like anywhere you work these

days, you know, you just have to be prepared for something to change at any moment, especially if you plan on having a career in tech. I mean, you have to be prepared to pivot. You know, it's it's

sometimes you have to be prepared to pivot into an unknown direction. So,

what was I going to do with this news? I

mean, shoot, I had I couldn't sit around and worry about it. You know, I had to lead a team. I had to develop relationships. I had a new product to

relationships. I had a new product to learn. I mean, you know, people were

learn. I mean, you know, people were counting on me to provide guidance, direction, and leadership. Did I feel anxious? Yes. I mean, you know, I was

anxious? Yes. I mean, you know, I was cons. Was I concerned about starting a

cons. Was I concerned about starting a new job? Uh, you know, just as a company

new job? Uh, you know, just as a company was going from public to private, you bet. I mean, I just left the place.

bet. I mean, I just left the place.

Well, I knew everybody was about to get laid off and they did. So, I was very concerned. But you know as I think about

concerned. But you know as I think about it you know this talk isn't about how to just survive when a company goes through some type of significant change. No uh

this talk is for any researcher at any company who is facing new change just any change at all like whatever they may be budget cuts reorgs leadership change.

This talk is about how to thrive no matter what the change you're in is because let's let's face it this is the honest truth. Researchers around the

honest truth. Researchers around the world, everybody in this room who's a researcher and everybody on the live stream, the truth is researchers around the world are being asked to do more

with less. This is real talk. This is

with less. This is real talk. This is

the way it is. Our discipline, we've seen it recently. We're not immune to the decisions that companies have to make as a result of macroeconomic

pressures. We either adapt or we get

pressures. We either adapt or we get left behind. That's real. E and I read

left behind. That's real. E and I read somewhere either you step into the change or the change is going to happen to you. So this talk is for researchers

to you. So this talk is for researchers no matter the situation how you can pivot to become essential strategic drivers of company success and I hope

you walk away with some ideas today for how to do that. Now but spoiler alert nothing I'm going to tell you is new. I

have borrowed all these ideas. Okay. So

now let's go back in time a little bit.

In the first few months that I was at Zenesk, a couple of things stood out to me about how our UX research team was operating. First, uh, while some of our

operating. First, uh, while some of our research programs were aligned to key company initiatives, most were not.

That's a problem. Nearly all nearly half of our research was tactical. You've

heard a lot of people this week talking about that. We spent a lot of time

about that. We spent a lot of time working at the end of the product development life cycle at the validation stage. Now if you are a research team

stage. Now if you are a research team and you want to have max influence working uh the majority with your majority of your work being at the validation stage that's not where you want to be. The second thing I realized

was that while our researchers were connecting with some of our key stakeholders we weren't consistently connected to all of them. And what I mean by that is that we weren't

connecting with them in ways that fostered real true cross functional collaboration and partnership. I mean,

we didn't we weren't leveraging our talent, skills, and expertise in the right way. So, in other words, it's like

right way. So, in other words, it's like we were in meetings, but we weren't in the right meetings with the right people at the right time. And that's important.

And this minimized our ability to our opportunities to provide strategic guidance to our stakeholders such as, you know, helping them figure out what problems to solve for our customers.

Now, look, going from public to private did not cause any of these issues. I

mean, they were already there, but it forced us to fix them because, hey, look, you know, when a PE firm requires a new a new company, the one thing that you can count on is that focus sharpens

fast. That's what happens. Now, we were

fast. That's what happens. Now, we were fortunate, Zenesk, to be acquired by a PE firm that is uh focused on long-term growth, you know, and building for the

future. Very excited about that. Not all

future. Very excited about that. Not all

PE firms work that way. So, but over the course of the next 12 to 18 months, our company began to focus on several key priorities and we had to shift our

research team's operations and how we worked to map directly to these priorities.

But we faced a critical question that I'm sure everybody in this room is facing right now or you will soon face.

And that is how do you maintain high quality impactful UX research when resources are tighter and expectations are higher? How do you do

that? You know, it's like walking on a

that? You know, it's like walking on a tight rope. It is literally like how can

tight rope. It is literally like how can you expect to have more impact when you don't have new headcount? you have flat year-over-year software spend, flat T&

like we could not work any harder. Our

team couldn't work any harder. We had to work smarter. So these changes that were

work smarter. So these changes that were happening at the company, these new constraints, you know what they did for us? They became catalysts for our

us? They became catalysts for our reinvention. That was very exciting. So

reinvention. That was very exciting. So

you know, I say this pivoting is just part of the maturing process of a UX researcher. It's part of the maturing

researcher. It's part of the maturing process of a UX research team. This is

where we were. we had to lean all into it. So today I'm going to talk about

it. So today I'm going to talk about three shifts that we made. Now this

isn't a comprehensive lift. We list we instituted a number of strategies to help us operate in a better way. But I'm

just going to share with you the most important ones. All right. First, we

important ones. All right. First, we

first aligned our research to company OKRs and product team OKRs. All right,

here's how we did that. Now, we couldn't continue, you saw this slide earlier, we couldn't continue to operate like this.

It just wasn't sustainable. We needed to pivot from this to something more like this. And let me show you how we did it.

this. And let me show you how we did it.

Now, this is an illustration. This is

our research team and the areas where each researcher is aligned. These aren't

real Zenesk researchers. All right? And

these aren't AI research. These are

actually real photos of real people, but just not our team. Illustration purposes

only. All right? We were very intentional about how we constructed this alignment. Uh we prioritized our

this alignment. Uh we prioritized our work. And these these aren't even the

work. And these these aren't even the number of OKRs that we have at our at the company level, but we prioritized our work to have the biggest impact on our company's most important initiatives. OKR 1, 2, 3, and four. So,

initiatives. OKR 1, 2, 3, and four. So,

our researchers, as you can see here, are directly aligned to them. Now, we

remain flexible and we made adjustments throughout the year given the needs of the business, but this really helped us really solidify where we were aligned in the organization. So, we created this

the organization. So, we created this map and it really truly helped us to focus. Now, Second, during the roadmap

focus. Now, Second, during the roadmap planning process, which are many I think many of your your companies may have the product organization team, they then create their OK their level two OKRs and

their key initiatives. We are involved in that process and then we align our UX researcher to partner with all of the cross functional partners for each level

two OKR. and not shown here as the PM,

two OKR. and not shown here as the PM, the design lead, the um product marketing manager, all of those folks.

They're not listed here, but um what we did was we created this alignment map.

We shared it with all of the shared it in all the appropriate Slack channels.

Of course, we shared it at all the all hands meetings just so everyone knows who the researcher is, what they're working on, which initiatives, and all of the uh against all of these OKRs. And

this really changed the game for us. You

know, researchers were in now in all the key meetings. All right, they were

key meetings. All right, they were invited now to all the key meetings where product strategy is discussed.

They had one-on- ones with all the cross functional leads, right? And then they um we began to build a system that was really working for us. Now, notice when

I said I use the word partner. Partner,

we partner with our crossf functional teams. We do not support them. We do not provide support. Now you say, "Oh, let's

provide support. Now you say, "Oh, let's these are semantics." But this is real.

It's very important because words do matter. When you

matter. When you support, you know, they can call you up and it's reactive. But when I think Brad talked about this, when you when you partner, you're strategic. You are

working alongside them. And there is a difference. So we proactively involved

difference. So we proactively involved uh we were proactively involved in the key conversations. Now you know now

key conversations. Now you know now these were relatively small but targeted changes but they made all the difference

for us. So now every project had to be

for us. So now every project had to be tied to an OKR. No OKR, no project. Can

I say that again so everybody in the room understands this and on the stream?

No OKR. No project. Nobody on the stream heard you. No OKR. No child. That's it.

heard you. No OKR. No child. That's it.

Now I know this is extreme. This is this is provocative intentionally. This is a good default position to be in. This is

what we had to be in. We are a team of 12 small research teams striving to have max impact. You know, you you have to

max impact. You know, you you have to draw the line somewhere. You can't fit 10 pounds of sugar in a 5B bag. It

doesn't work that way. You've got to draw the line somewhere. So, in these last three years, we've just been very strategic in how we do this. Now, before

you say it, this is too extreme. I know

that there are exceptions. All right.

While we focus, we make sure our work is aligned and focused on OKRs. We've built

some flexibility into this because, you know, let's face it, research is not linear. We know. Excuse me. We know that

linear. We know. Excuse me. We know that we do take on projects that may not map cleanly to the OKRs, you know, like when you know, new questions pop up or emerge

doing analysis or workshops uh and it may reveal some critical gaps in our knowledge that we don't know. So yes,

we'll partner with that. You know, early discovery work for zero to 0ero to1 projects. Yeah, research needs to be

projects. Yeah, research needs to be nestled into those conversations.

Time-sensitive launch. Yeah, sometimes

the product team really needs some quick uh insight to inform a product uh decision. So we stayed flexible, but we

decision. So we stayed flexible, but we always asked the question, which OKR could this research ultimately help drive? All right. So, we

aligned our work to the company priorities and OKRs. We're now

partnering very closely with our cross functional partners. We're in the

functional partners. We're in the meetings we supposed to be in. But then

we started thinking like how can we truly elevate how we operate strategically within the organization.

So, here are two ways that we do, two things that we installed. And again,

these aren't new. This exercise is called I wish I knew. And we wanted to um work we were already working now more closely with our cross functional partners and we wanted to find a

scalable process that we could deploy when product teams were sort of struggling to articulate critical knowledge gaps that they had about what their customers are thinking. So we

launched I wish I knew not my idea borrowed this from another company I work for. We pulled uh the cross

work for. We pulled uh the cross functional leadership together and asked them one basic question. What do you wish you knew about customers experience doing, I don't know, experience using

our messaging product? What critical

customer questions do you still not know the answers to that could help inform a product decision? So, this is a really

product decision? So, this is a really easy exercise. It's actually most

easy exercise. It's actually most effective at the beginning of roadmap planning. You know, at the or at the

planning. You know, at the or at the start of a project that, you know, it's an ambiguous space. um a lot of uncertainty around who the customer is that you're designing for or um you know what the problem is that you're actually

designing for. I know you can't you

designing for. I know you can't you probably can't read all of the words on this slide. This is just for

this slide. This is just for illustration purposes. So you see here,

illustration purposes. So you see here, you know, you got a bunch of post-it notes together. It's design thinking

notes together. It's design thinking 101. You give your uh your your

101. You give your uh your your teammates some some instructions. Takes

them about 10 or 15 minutes to jot down their questions. And of here they go.

their questions. And of here they go.

They jot down the questions. and the

researcher. You're the researcher. You

do like a little mini thematic analysis and you move the questions. You identify

some common themes. Again, design the sign thinking exercise. Everybody votes

on it and then voila, you can now prioritize them. Now, this takes about

prioritize them. Now, this takes about an hour to do, maybe an hour and a half.

And it's nice to do it in person if you can if all your team is together, but you know, we've been doing it virtually.

We use Figma or Mirror or some some other collaboration tool that works. Um,

you can also do it async like we are globally distributed. So we have folks

globally distributed. So we have folks in Singapore all the way to Estonia, Serbia, Berlin, everywhere. So um

sometimes we have to set it up as a an async exercise and it works. Um you can have a lot of impact with this. It's

just it's really interesting and it surprised me how much impact it was having especially you know when you're joining a new team as a researcher. You

know this really works then um you can create it's a way to create sort of jumpstart your learning about the known unknowns with your team. you know, this is just it's just one of many um methods

that you can use to present yourself again as a strategic partner to your stakeholders. The second thing we

stakeholders. The second thing we started focusing on is our writing. We

began incorporating what I call POVs into research reports. Now, let me explain this. It's another thing I

explain this. It's another thing I borrowed from from another company I worked for. As researchers, we're so

worked for. As researchers, we're so used to writing recommendations, findings, and recommendations. And when

you think about a recommendation, it's based on answering the question that you formulated when you designed the research program. Now,

when you think about a POV, a POV is what you think should be done as a next step, not just based on the research that you conducted, but on all the information that you've come to learn

about the focus of the research. So,

your POV brings together all of the various perspectives that you've covered like the behavioral data, all the competitive trends, market research, historical data, anecdotal stuff, past

research. Your POV connects it all

research. Your POV connects it all together to a point where you're able to assert an opinion. It's a POV, an opinion about what your product team should or shouldn't do, what they should

build or shouldn't build, what they should ship or shouldn't ship. This made

a huge difference. This is just another madeup example of what it looks like.

Um, it's simple, but it had a lot of impact. This is like our post research

impact. This is like our post research elevator pitch, I call it. It's the

first slide of our decks. And, you know, it's simple. Create an image. We've got

it's simple. Create an image. We've got

the research POV right there in the middle. Some supporting insights. Again,

middle. Some supporting insights. Again,

this is a madeup example. Don't laugh at it. It's it's made to be funny. Uh,

it. It's it's made to be funny. Uh,

we're we're still maturing in how we do this, but we want to make sure that our POVs are confident, clear, grounded in logic, and it connects all the dots. So

this I wish I knew exercise and writing clearer, more direct POVs were two ways that we uh really tried to elevate our influence at at Zenesk. Now another

audience participation activity. By show

of hands, how many of you are on a team of five or less? Okay. Wow. Okay. Now, how many of

less? Okay. Wow. Okay. Now, how many of you are a team of one? I know there are a few of you in here. Okay. So, um,

those of you who are on the team of one, uh, do you ever you ever recognize this when you look at all the people that work that work at your company and they're coming to you, do you ever feel like everybody's coming to you with their hands up asking you a question?

Yeah, if it may feel like that. I mean,

we're a team of 12 and we we do feel like this. Um, and they're asking you

like this. Um, and they're asking you one question. Has your team ever

one question. Has your team ever conducted a research project on blah blah blah? I'm like, really? Like this

blah blah? I'm like, really? Like this

true story. Some someone a couple months ago asked me this. They said, "Hey, Reggie," they slacked me. Um, "Has your team conducted any research on how our customers are using

AI?" I'm like, "Oh my god." I mean, the

AI?" I'm like, "Oh my god." I mean, the snark in me really wanted to say it. I

wanted to say it, but I held it back. I

held it back, but I will say, "No, you know, we we're having capacity issues.

We'll get to that next quarter." But I didn't say it. I didn't I didn't say it.

But it's, you know, it happens. You

know, you feel like uh you have all these questions that are bombarding you.

And three years ago when I joined Zenesk and I joined my team, you know, we you know, we have 200 plus scrum teams and say that there's no way that we can sit there at our UX ask UX research Slack

channel and monitor it all day. We

can't. We do not have that capacity.

Now, these are well-intentioned questions from well-meaning PMs and designers. All right? Um and we want to

designers. All right? Um and we want to help. But three years ago, we were just

help. But three years ago, we were just pulled in too many directions and it was distracting us from the work that we were conducting that was aligned to OKR.

It's like this wasn't an efficient way to operate and we weren't being strategic. So we needed to do two

strategic. So we needed to do two things. We needed to scale the

things. We needed to scale the intelligence that we already have and then build a program to increase our research capacity and enable non-ressearchers to conduct research.

Okay, before I know I hear some groans.

I know you're groaning. I know you're groaning. Look, I wasn't sold on it. I

groaning. Look, I wasn't sold on it. I

wasn't bought into it. Trust me, I was democratizing research. No, but I'll

democratizing research. No, but I'll show you how we did it without minimizing our research influence and expertise. So, I'll show you that in a

expertise. So, I'll show you that in a second. So, let's start with the scaling

second. So, let's start with the scaling our knowledge. And many of you, I know

our knowledge. And many of you, I know you've done this already, but think three years ago, we really weren't we, our operation wasn't to the point where where I thought that our research

repository was where it needed to be for it to be a a true searchable comprehensive database. I wanted to make

comprehensive database. I wanted to make sure every single research report we had conducted to date was in this database.

Now, um, this isn't what it looks like on Confluence. I'm trying not to throw

Confluence. I'm trying not to throw shade. I'm really trying not to throw

shade. I'm really trying not to throw shade. I may had to make this one

shade. I may had to make this one up, but you can see this is what we do put out. Now, we have a comprehensive

put out. Now, we have a comprehensive database and it has all the information in our research POV, top insights. We

also record short videos if when we can um with our TLDDR and it that that really works really effective especially with senior leadership who they don't want to read a report. So a two and a half minute three minute video with a

TLDDR that summarizes the POV. Oh, it

works like a charm. It's awesome. We

wanted to make sure as I said that every piece of research that we had conducted to date is available and searchable and and accessible to everybody in the company. Now, we're exploring ways to

company. Now, we're exploring ways to attach this to our gen a gen AI system so that it can automate through Slack.

We're not there yet. Trying to

accelerate that process. I think there's more that we can do there, but we're going to get there. So, second, let's talk about democratization. Like, I know it's a hot topic, pros and cons on both

sides, but you know, we're a team of 12.

I had to step back, think strategically, and I believe now if done right, you can have a successful program without diluting the skills and expertise of your team. Now, I know this is not a

your team. Now, I know this is not a very pretty look, but I'm going to share with you this is our uh DIY training program that we created. And what's

different about ours that from others that I have seen is that it resides in our company's learning management system. All right. So we worked with

system. All right. So we worked with that team to build a training five training modules as you can see um that uh there are like 20 30 minute videos

across these topics and in order to access our user testing license. So we

provision all the licenses if somebody that's a non-ressearcher wants to um conduct research themselves DIY they have to go through this training course and be officially certified and you can't get any more official than the

learning management system. I this is the same system that company that our company uses to do the code of ethics training um your harassment training, your fishing scam training, you know, all that stuff. You have to go to this

in order to get certified to use our research systems and tools at Zenes. And

so that's what we put enough friction in there. So you got to be serious. You

there. So you got to be serious. You

can't just trip and fall into doing research uh with our tools at at Zenes.

You can't do it. Now this is re basic research 101. All right, that we're

research 101. All right, that we're teaching them in this training module.

We're not teaching them how to build surveys and do fancy multivariate regression analysis. That's not

regression analysis. That's not happening. We're not going to do that.

happening. We're not going to do that.

This is basic user interview kinds of stuff. How to ask better questions, how

stuff. How to ask better questions, how to avoid big mistakes, and most importantly, when to call us. And what's interesting is that some

us. And what's interesting is that some of them try to get certified and they're they finish up and and they start the research and they go, "Oh man, I can't

do this." and and so I'm like hey you

do this." and and so I'm like hey you know now you see so you know don't think that you know you can just as I said trip and fall into being a researcher no it doesn't work that way but most of the

projects are simple they're validation exercises you know things that PMs can do and and we're okay we keep an eye on it but let me tell you something enabling some designers and and and

intent like well-intentioned PMS to do it really multiplied our impact I mean it was amazing so the first year we saw a 52% increase in the number of foundational research projects that our

team was able to conduct like I have eight at that time I had nine IC's and it was amazing and we spent now twice as much time in the discovery phases of the product development life cycle huge win

for us and the majority the vast majority of those research requests that come in uh we were able to deflect them to the self-service channel and it

worked really really well so what did we do to pivot we aligned our work to OKRs.

We elevated our strategic influence and we created more access to the information that we have and we enabled teams to self-s serve on low-risk research projects. Now, look, you can

research projects. Now, look, you can all do this the same. You you probably have already done some of these things, but it's about having focus and that's what we did. So, I'm just going to say this now. If you haven't uh if you don't

this now. If you haven't uh if you don't remember anything else in this talk, remember this. Stop working on projects

remember this. Stop working on projects that are not tied to company priorities.

Stop. Just stop today. They don't

matter. No. OKR. No. There you go. All

right. If it doesn't drive Listen, if it doesn't drive impact, it doesn't deserve your time. It doesn't. It doesn't

your time. It doesn't. It doesn't

deserve your effort. Tie your research programs to the work that matters the most at your company. The work that drives your company's business strategy.

Everything else is noise. Now, our story at Zenesk is still being told. Like,

we've had a lot of impact. I'm excited

because the the projects that our CEO has shared that he worries about every single day, I'm proud to say we have a researcher partnered with the product teams on every single one of those

projects. That's super exciting. You

projects. That's super exciting. You

know, I'm proud of that. But we have more work to do. Um there's more impact that I think that we can have and we will pivot as much as we need to in order to make that happen. We're still

owned by private equity. Yeah, we're

still facing tight budgets. We're still

navigating reorggs and leadership changes and everything else. Like change

is happening. But you know what? These

changes didn't break us. They're not

breaking us. And they don't have to break you. They have sharpened us and

break you. They have sharpened us and they have forced us to really focus. So

uh I will say this as I close that none of what I have shared with you today is going to help you is going to guarantee job security. Right? There's no such

job security. Right? There's no such thing as job security. But what I hope it does is give you something more um powerful and that's career confidence.

You know, confidence to be creative and resilient when things are unclear like it was for me three years ago. You know,

to to stay grounded uh when everything seems like it's changing and shifting underneath you, to know how to work, to learn how to think strategically in your roles as researchers, you know, to know

how to make your research matter. That's

how we've thrived at Zenesk and that's how you can too. Thank you very much.

This is how to reach me.

Let's take a seat for a sec. That was brilliant. Really

sec. That was brilliant. Really

invigorating.

Um I have many thoughts, but go ahead. I

would love no research. No OKR, no research. Talk about that. No, I I love

research. Talk about that. No, I I love I love the journey that you're on.

You're still on it. And I I love they also open about that, right? This is

what you found to be effective. Um and

we're going to see how things unfold.

But thank you so much for sharing that journey. Let's take one question from

journey. Let's take one question from from the audience because I'd love to I'd love to hear one. Um, anyone have a question for Reggie? Because if not, I do.

Going right there. That was so comprehensive.

Hi. Amazing talk. Um, I am curious to hear a little bit more about the zero to one initiatives on the team. And so

thinking about not the OKRs of today, but maybe the OKRs of the future and how you equip your team to think about um

inspiring what metrics we might the company might care about maybe in one or two years. There are opportunities that

two years. There are opportunities that we have to work on those projects. They

spin up in different ways at the company now. I mean the companies right now is

now. I mean the companies right now is really focused on some very specific lines of work and what we what I try to encourage the team and I coach the team

to do is just to be flexible and understand how to identify when a zero to one opportunity could come to play.

So um what I I think the team now um is is is really excited when you know the research sort of takes them into kind of

an unknown area and you know we they they are focused on really trying to help the product the product leadership

the design leadership um just figure out what those um problems the customer problems could could be. I mean

obviously every company's focus is on AI now but we have some smaller things internally that I think are interesting places for us to uh to create some white

space. So um we don't have necessarily a

space. So um we don't have necessarily a playbook for how we do it. It's just to it's just for the researcher to be um open and under and cognizant of the

opportunity when it presents itself that hey this is the research that we've come up with but boy these other areas could be interesting for us to advance our knowledge and also advance what we could

potentially do with the product. Um so

it's just very important just to have that awareness and I think once we start there you can then figure it out after that.

That's lovely. All right, I'm sure many of us will have many more questions for you, Reggie. Uh, thank you so much for

you, Reggie. Uh, thank you so much for that brilliant talk. Got to give a round of applause to Reggie. Thanks so much.

Thanks, buddy. There you go.

Okay, folks, we are down to the last but definitely not the least session. You know, something we've been

session. You know, something we've been talking about, I think, all through the week in various forms is how sort of generic outputs and experiences are just

not the thing anymore, right? And what

I'm excited to learn about today from Jennifer is how we can use this real data to get bit better data. So, without

much further ado, Jennifer, why don't you come to the stage, give a warm welcome for this last session of the day. Thank you.

day. Thank you.

Thanks. Hi, I'm Jennifer. I've somehow

landed here to be your last speaker of the day, and I hope I can hold your attention with a topic that I'm very passionate about. Um,

passionate about. Um, prototypes. Prototypes are probably one

prototypes. Prototypes are probably one of the most common things that we use in our practice, especially as UX researchers, as our stimuli in our sessions, especially in pre-launch

scenarios.

However, so often we are overlooking just getting the prototype that our designer handed off to us that might be like pixel perfect, the ideal experience and thinking really hard about things

like the protocol, the recruit, you know, the moderation guide, and not thinking about how the information that's actually within our prototypes shapes how we're doing those sessions and the insights that we're getting and

bringing to our teams. I really think this is an area that we need to think about upleveling ourselves in as a practice. Um, and I'm going to give you

practice. Um, and I'm going to give you so many practical tips about that today.

But first, I'm going to talk about one of my other passions, hamburgers. I love hamburgers. Have

hamburgers. I love hamburgers. Have

since I was a kid, specific to a Smashburger, Beepsburg out here in San Francisco. If you're looking for some

Francisco. If you're looking for some place to eat tonight, think it's great.

Um, and I thought that I would use this example for you today, um, of what another thing that I will probably do this evening, which is like door dash a burger to my place after a long week of

learning, um, for a special treat. Um,

and I wanted to take you on a journey with me as I search for a burger and how the data within the stimuli shapes my reactions. So, let's begin. All right,

reactions. So, let's begin. All right,

let's imagine I'm the research participant here and I am shown, "Welcome to Caliber Burgers." Caliber

Burgers looks delightful. They have

invested in their marketing photography here. This looks very much like

here. This looks very much like something like my designers would pass me if I worked here. Like, you know, wonderful. I will go onward. Let me see

wonderful. I will go onward. Let me see what this has to offer. Oh my god. This

burger is $25. I mean, I am like really into

$25. I mean, I am like really into burgers, but like $9.99 was kind of more what I was hoping for for like ordering to my doorstep. Maybe $25 would work in

like a in dine in sort of experience, but this is not for me. I must move on.

Let's go to the next example. Jenny'sburg. Great

example. Jenny'sburg. Great

name. I though, however, see Jenny's burgers. It just this isn't speaking to

burgers. It just this isn't speaking to me. I can even just tell with this cover

me. I can even just tell with this cover image, which is probably taken with an iPhone, that like this is like some lackluster looking kind of French fries.

This burger looks a little sad to me. I

I don't think that this is for me. The

one thing that is actually catching my eyes is somehow this is the number five burger in San Francisco. And I just really don't believe that to be true. I

want more information, but not about how I'm going to order this burger tonight.

Final scenario. Let's imagine a researcher

scenario. Let's imagine a researcher presents me with this sushi. Maybe I

want sushi tonight. Thing is, I don't want sushi tonight. I don't eat fish.

So, if I were presented this in a research session, I would probably come up with something very peopleleasing like, oh, you know, like two for one for fish sounds like kind of a terrible idea from what I've heard about freshness of

fish. Um, and uh just try to move on,

fish. Um, and uh just try to move on, answer the next question. Okay,

hopefully you get my point here. The

things that we are showing to our participants in research sessions, the images, the prices, the contents sometimes which are like Laurum, right?

um are not doing justice to seeking the insights that we're trying to get. We

really need to be more strategic about the data that we're putting into our prototypes. The reason for this is

prototypes. The reason for this is really, especially in a pre-launch scenario where you're not, you know, showing public information yet or able to readily bring in um information that you might have live on the site is

because we're bound to this world of hypotheticals. Okay? and we're trying to

hypotheticals. Okay? and we're trying to give insights on what that reality is going to be for our users come that faithful launch day. We're bound in the world of hypotheticals. We're not really necessarily getting at the meat of those

insights. Um I kind of come to think of,

insights. Um I kind of come to think of, you know, sometimes we just have to use mediocre data, but mediocre data can lead to mediocre results. I think that we've all seen this. If you show

someone, you know, a beautiful caliber burger for every sort of single experience they'd see and everything has the right price point, you're not getting the richness and texture there.

You're getting probably mediocre results and in worst case scenarios, you're actually getting misinformation. I

think, you know, people have been, myself included, have been in things where you might be in such a hypothetical sort of scenario at the research phase that you and you're not putting enough real information that you can actually mislead your team. Yikes.

Bad situation. We don't want to be misleading our stakeholders. And that's

why I'm trying to prompt us to do something further. But of course, if

something further. But of course, if this was so easy, why aren't we doing this today? It's like, of course, this

this today? It's like, of course, this is so easy. Why aren't we doing this today? I have several excuses that I

today? I have several excuses that I have said and would like to share with you and then share with you how you can get past them. The first that I've said

them. The first that I've said definitely before, it's like, I don't have the data. You know, the data lives in some like very off there table with my organization, and I I just don't know where it is, and I'm not a data

scientist.

Sorry, I'm not a designer. You know, you don't want to put me in Figma. It's

really gonna mess things up. Um, so

that's that's great, but I don't know how you're going to ask me to do like eight distinct prototypes for eight different participants in next week's sessions. And then one that ties to both

sessions. And then one that ties to both of these, I don't have the time. Like,

of course, none of us have time, right?

I don't have time to do new tasks or learn new skills. Um I uh think that we actually can though work differently and I think it's time to work differently.

Um and in the spirit of working differently um I am going to show you a timeline of kind of like how you a typical research project might go in the

shape of a burger.

um I'm really leaning in here um to to how you can have some practical tips in your next research project to be thinking about how you can actually do diff things just a little bit

differently to overcome those barriers that I just spoke about. Okay. So first

let's start with planning. In the

planning phase like any research project it's important to bring the team together. And when I say the team, I'm

together. And when I say the team, I'm specifically thinking of like when you're talking with like your product manager, your designers, your data scientists, your engineers, bringing them together, you know, not just about the research brief, but about the

importance of why you believe you need to have real data in this study, why that's going to actually change your insights and results. Um, little spoiler here, this is also the time that you can try to get the resources of folks to

help you with that that might unblock you on things like I don't have the data, I don't know where it lives, or I'm not a designer. this is the time to start getting that sort of buy in. Okay.

The other thing that's really important in the plan phase is like locate the data that matters. Um I I completely understand you know creating eight

custom prototypes for an individual study potentially more um is a tall order and so you need to like anything locate where you're going to focus on the data that's going to matter to your

particular study at hand. Um, and so you know, I was searching for dinner for tonight, but you might be doing several different things depending on whatever you're, you know, project you're working

on. So maybe you are working for like a

on. So maybe you are working for like a budgeting app or something in finance.

Maybe it's about a search query. Maybe

it's about browsing social media content. These are all different sorts

content. These are all different sorts of examples where you might need to bring in different data. What I have found there are a couple of kind of specific things that repeatedly come up

that prompt different responses from users in sessions and those are prices. The price point of something

are prices. The price point of something matters to you in context if you have a budget, right? So like I don't want a

budget, right? So like I don't want a $25 burger tonight. Maybe a different time, but I want a $999 burger tonight.

People respond really differently when they have skin in the game and a budget in mind for a specific like task, action or checkout. So, this is a time when

or checkout. So, this is a time when it's really worth investing in. Another

one is like search results. If I'm searching for a burger,

results. If I'm searching for a burger, I don't know or want to talk to you about sushi. Like, it's not you need to

about sushi. Like, it's not you need to be actually searching for something where you can give feedback on the results that would actually be meaningful to whatever you're like the task that you're doing.

Um, and then also user generated content. And this could be, you know,

content. And this could be, you know, kind of I showed like an Instagram post.

This could be like an Instagram post where, you know, maybe they're not looking at quilts. Maybe they're

actually like really into like Taylor Swift's latest or something like that.

You want to speak to whatever they are going for. Um, it can also be not

going for. Um, it can also be not userenerated content in the like sense of social media. It can be the user generated content from like steps on a walking on a dashboard for like a health

app or like the prices that you would saw in that budgeting app. That's still

kind of generated by that person and they're going to react to it differently when they kind of are indexing on content that's meaningful to them. I'll also call out this isn't for

them. I'll also call out this isn't for every study. Um, as should be said, this

every study. Um, as should be said, this is a heavy lift of investment to get better insights and isn't necessarily applicable if you're like reviewing a homepage that isn't going to be variable

by user to user. So, we're all professionals here. I'll leave it to you

professionals here. I'll leave it to you to decide, but now would be the time before you go further. Um, next, recruiting. This is a

further. Um, next, recruiting. This is a really important phase because you're going to need to I had a digging little emoji here that didn't come through for

you. Sorry, guys. Um, but you're going

you. Sorry, guys. Um, but you're going to need to pull the data per participant. Um, because that's really

participant. Um, because that's really important. We were remember how I was

important. We were remember how I was talking about like the tables that you might have at your uh at your company.

You might have like search history of people or you might have what sorts of like prices or things you might be looking for and different things. You're

going to want to pull that data per participant if you can. I often partner with a data scientist for that because that is not always my area of expertise to dig really deep into those tables to get that. I understand sometimes those

get that. I understand sometimes those resources aren't available. But there is always something at your disposal and that is the screener. You can always ask for data in the screener. There's no

excuse here, right? We can all do this.

Um so you can ask for this data in the screener by just asking simple questions. You don't need to get the

questions. You don't need to get the exact data all the time. You could just say like what's your budget? so that you then input into those designs a realistic budget for that person or an unrealistic budget like depending on

what you're trying to do here. You could

say, "What have you been searching for online?" So you actually show them

online?" So you actually show them relevant content for stuff that they're actually have been searching for and want to engage with and are going to have an opinion on. Similarly, what

sorts of posts have you been interested in lately and really zero in on what they're going to respond to and have an opinion about? Um, another reason I

opinion about? Um, another reason I would say even if you can pull the data, I would recommend still asking some of these questions because you are going to in the proctor phase present them with

something that's quite personalized and that could be like borderline creepy if you haven't put together the fact that like we do have all of that data stored on you and we're presenting it to you.

It just brings a little bit more of kind of bringing the participant along on the journey um that you're going to be showing them a very personalized

experience. Next, design. This can be a

experience. Next, design. This can be a really intimidating phase. You need to plug all this data that you collected into the design and if you're like me with Figma, it's like not, you know,

going to go very well. So, um and I'm often passed something from my designers that is beautiful just like, you know, these ones. They're they're launch ready

these ones. They're they're launch ready sorts of images. And so it can be really it's like okay so uh am I going to like copy and paste all of these or like designer who is working on 1 million

other things could you please like spend two whole days creating eight custom prototypes for my study. That's usually

not something that's available to you.

Fortunately there are actually tools that can help you do this. So like one that is this is one that I've just like pulled Google Sheets sync. This is a Figma plugin. There's tons of Figma

Figma plugin. There's tons of Figma plugins and things like this. I

encourage you to like push yourselves and your designers to find solutions to this. In this plugin, what you can do is

this. In this plugin, what you can do is you have your Google sheet with your information and your designer can create just like one Figma prototype that they then have the different components for

your different data points and it's as simple as kind of like refreshing to the row that you have that you need. I've

used thing tools like this to make like designs very quickly with my design designers without ever touching Figma personally. Um the other thing is I

personally. Um the other thing is I think we so often are like oh you know we're at this like high fidelity phase but you can actually write the data on paper and I've done this before. I think

we often say like oh paper is for the early concepting phase but you can use paper really any time. You could also do this like on a Google slide, but I've done in studies. I've actually taken

the, you know, the beautiful Figma design, printed it out. You can use just a little white out for whatever that key data point is like a price or, you know, whatever that sort of thing might be,

and you can just write it in. It'll give

you that more realistic response that we're looking for without totally doing disservice to the fidelity of the design. Finally in

design. Finally in proctoring um so when you're proctoring it as I mentioned the one thing I would say is like prepare your participants a little bit I don't have this as step so

plan per participant and prepare your participant prepare your participant a little bit by saying like when they come I usually when I transition into the individual session I'll say something

like we've actually customized these prototypes just for you kind of based on their conversation because it can be a little bit jarring to see potentially personal information you know immediately on someone else's computer

screen when you're going into sessions.

So that's one part of planning for participants. Another thing is is every

participants. Another thing is is every single session is actually going to be a little different, right? Because you

have all these individual prototypes. So

like participant one might be a burger searcher, then you have a sushi searcher, then you have someone searching for pizza. It's super

important to stay organized here because you don't want to show the wrong person the wrong thing. Um, so if it's in Figma, I've just had my designers do something like a simple kind of dashboard for me that's on a separate

page that maps to like participant one and I know participant one's burger.

Okay, participant two, oh, okay, there's sushi person. Participant three is like

sushi person. Participant three is like so you're able to do all of that. It's

also super important if you're doing things that would actually leak personal information participant to participants for privacy reasons like paramount to invest in keeping these things straight.

If you're doing paper, I just keep little physical folders for each of my participants. I feel very lab coat like

participants. I feel very lab coat like with that scenario. Um, so this is just how you can go ahead and make sure that you stay organized during this

phase. Finally, moving on to insights.

phase. Finally, moving on to insights.

the insights that you're going to be, you know, doing synthesis and things like that are pretty similar that you would be doing for any study that you might be doing, but you're going to probably be getting richer insights, right, that you're going to be getting

since they had these real prototypes that they were interacting with. And um

what you're actually going to do, which I am not usually a proponent of, but I am in this case, is I I would love you to call out the method in some way, shape, or form until it is common

practice that we're all using real data in our prototypes because your partners might not have followed along on your journey of all these sessions and so they're just seeing whatever your output is. There are different ways that you

is. There are different ways that you can do this. you can, you know, just show a video and they'll with the real data being in there and I'll just call out afterwards. I'll say like, you know,

out afterwards. I'll say like, you know, this participant is had this sort of experience and this is actually with the real data that they will be seeing come launch. I get a whole different reaction

launch. I get a whole different reaction from my stakeholders of like, oh wow, you know, when they know that it's the real data, it's not placeholder data that they're seeing. And it's also a

great time for you then to kind of close the loop with those partners at the beginning who might not have been as eager to partner with you because they didn't have time or whatever. They will

find time next time when they're impressed by the insights and they're bought into this process. So this is the way that like by by socializing that we're starting to make this a practice that we regularly put real data in my

prototypes with my you know stakeholders. I'll say ah we could you

stakeholders. I'll say ah we could you know do that study in two weeks with placeholder data. give me three weeks,

placeholder data. give me three weeks, we could do real data and can you give me that data science resource? Give me

that designer for just like a day. Oh,

yeah. Oh, that's worth it to you. Great.

We'll do it. So, that's another reason why you're going to want to call out at this insights phase just like what was produced from what was put in to get more help in the

future. So, we have built a burger, you

future. So, we have built a burger, you guys. That is a lot of practical tips

guys. That is a lot of practical tips for you that I really hope you can use in future studies. Um, but if you're going to take away just one thing

here, I it's really don't be mediocre. Like, let's not I think that

mediocre. Like, let's not I think that the whole thesis of so many of these talks is different ways that we can be upleveling our practice. The specificity

of my talk has been about prototypes.

And for prototypes, what that means is upleveling our prototypes with the data that we're putting into them. So, I

really think that this is something that's quite achievable given the tactics that I spelled out. I'm very

passionate about this if you haven't been able to tell. Um, and so, please come up and talk to me afterwards or share your tips of how I can be implementing, you know, your tips on

these um, as we go forward. So, thank

you so much. Thank you to learners. Um,

and I will be having a burger this evening.

[Applause] Should I go to We I I took the burger away.

All right. Oh,

can we get Roy a mic? Another mic. Oh,

there we go. We got it. We got it.

Perfect. Um I love the burgers. Feel

like one is in my near future as well.

I one thing I was thinking very practically as you were talking was I know someone sitting here someone online thinking I want to bake this in um just

thinking about like the time investment for doing this well how what would you advise someone to like keep in mind when it comes to okay I want to experiment with this how should I be thinking this in terms of like how I invest my time my

research timeline everything because the first time you do something is always like okay 100% I think that's such a great question thank you um So I would say this is right if you're experimenting with any new method I

would use it on a project where you have maybe I have found that I've been able to take the time down on this because I do it repeatedly but maybe the first time that I did it it would take maybe

an additional week to week and a half to bake into your timeline. Um so that would be what I would say that also hugely depends on if you're lining up resources of help or whatnot. I would

also say then because if you're going to stretch your timeline anytime like don't do it on the most urgent project in the world. Um and and really I guess what

world. Um and and really I guess what I'm trying to reinforce with that insights piece is this is like this is a muscle. It's not something that you do

muscle. It's not something that you do once and stop. It's like something I laid out some methods that you know I thought was were practically accessible to everyone but I have done more advanced things with like prototype

engineers and stuff like that that our teams have started building as muscles.

And so what I really encourage you to do is find a project where you have the enough runway of at least like an extra week to week and a half where you have real data that will actually like matter

to the outcomes of your study and um and socialize it in the process because that's going to speed you up over time and start building the muscle of the team and the team will start my teams ask for this now. They say like oh can

we do this with real data? What would it take? I love that. We have some time for

take? I love that. We have some time for questions from the room here.

Um, one in the back.

Hi, thanks so much for this thoughtful talk. Um, I'm really struck by this idea

talk. Um, I'm really struck by this idea of real data and the richness of insights that you might get from presenting these really relevant, personally relevant um, prototypes to

your participants. One of the things

your participants. One of the things that I wondered if you had any thoughts on was how you ensure that your participants are getting um stimula that are kind of comparable to one another.

In other words, like how do you consider whether you're you're testing the same thing across participing? Thanks. Yeah. Um so uh

participing? Thanks. Yeah. Um so uh generally I'm trying to think of like where I've done that is I will think about I will recruit based on whatever

the criteria for like my team might be.

So, if we're looking for um let's say we're looking for a shopper with who's within a certain budget range or like in certain demographics, I'm going to recruit based on that still, right? In

the same sort of thing, which is going to inherently narrow in a little bit of like the experiences that they're exposed to, but there still is granularity in in a lot of those different vectors when you bring in real

data. So, I'm still I would say you're

data. So, I'm still I would say you're still recruiting based on your target

demographic, right? Um, and then I would

demographic, right? Um, and then I would like I might argue like we're always if if that's the target demographic that we would be targeting on and and building a

test plan off of and an AB test on, like that is exactly who you want to test for qualitative, right? Because as far as

qualitative, right? Because as far as qualitative is concerned, they're the same user. uh when you when it flashes

same user. uh when you when it flashes out in in test results and maybe you uncover something with real data that that isn't the same user, you know. So,

so that's kind of generally how I how I arrange that and I'm not sure if that exactly answers your question, but that's my thought process behind how I kind of how I target. And then if I saw some sort of variation that I thought

like was like, oh, our, you know, what we thought to be the truth isn't really true. Um that's a finding, right? that

true. Um that's a finding, right? that

is being input because you put real data into the target.

That's great. Any other questions before we wrap today?

There's one over here. Hello. Thank you for this. I'm

here. Hello. Thank you for this. I'm

like when can I do this next? So, I was already thinking of some of the projects. Practically speaking, when are

projects. Practically speaking, when are you telling your participants that you're using some of their like personal data? Is it during the phone screener?

data? Is it during the phone screener?

Is it in the screener itself? Is it

right before the interview? Like how

much time do you want to give them?

Yeah. To like consent in something like this. So, um number one, this all

this. So, um number one, this all probably depends on your company's data policy, right? Of like what data are you

policy, right? Of like what data are you able to even store and like share and stuff. So like certainly speak to the

stuff. So like certainly speak to the powers that be probably in your legal department about that. Um but the pro process that I've typically done is if this is all you know would be available

data to us and is things that would be shown to them in an interface in a very common sort of way. Um I I'm the first time I interact with my participants is also in the interview. So I do all of my

screening online. I don't I don't phone

screening online. I don't I don't phone screen. Um, I think of the screener

screen. Um, I think of the screener though as a way of that is my first interaction in many ways with the participant and if I'm asking them a line of question I truly am leading them

right a little bit. You're starting to lead them potentially even what the study is about to be. So you have to be like a little bit careful about that.

Um, but I generally the way I hold my sessions is I you know I spend usually like 10 to 15 minutes getting to know them just building rapport and that trust right that you initially need to

build with a participant. And then uh going into the actual part where I'm like, you know, I'm going to share my screen and this is what we're going to do or whatnot, I'll say, you were doing something a little bit different today.

We actually have taken some, you know, you replied in the screener XYZ and we've actually personalized based on that. Or, you know, if you're a customer

that. Or, you know, if you're a customer for a specific company where it's like very clear that you would have their data, like if you're a bank, like obviously you know some of their banking information. Just saying that out loud

information. Just saying that out loud though of saying like we're going to be reviewing this in a you know in a real and I usually say the reason why we've put in your real data is we want to make this as realistic so you can be putting

yourself in the most realistic scenario possible. So just also being like clear

possible. So just also being like clear about why we're doing it. We want real answers from you and we want real feedback and that same rapport building I do with all my participants of like thank you for your time because you're

giving me insights and and just being transparent.

That's great. All right, we have time for one more question before we wrap up for the day. So, is it going to be all right? Great talk. Um, I loved how you

right? Great talk. Um, I loved how you brought the hamburger theme throughout.

I have an operational question about the screeners. Sure. Um, do the screeners

screeners. Sure. Um, do the screeners reach the person with your personal name on it like as an introduction to talking with you personally or is it like coming

from some centralized team? like what is what does that communication look like given this very personalized approach?

Yeah. Um so I think that that's totally up to you. I'll give you like my my personal opinion. I almost always and

personal opinion. I almost always and you mean me personally like Jennifer Maples. Um I almost always have coming

Maples. Um I almost always have coming from like research at my organization um if they reply to it replies to my email if they would have any sorts of questions and I generally don't. And

then when I do the scheduling experience, it's usually coming directly from my company email address. So I I don't do I don't in the screening process make it any sort of more

personal. I keep it pretty standard cut

personal. I keep it pretty standard cut and dry and then start the kind of like journey of personalization. Also like in any

personalization. Also like in any sessions I find I find it's best right when you're rapport building in that first like 5 10 minutes with a a participant. That's really when I'm also

participant. That's really when I'm also even thinking of I always have placeholder in language but you're thinking like oh is this a very privacy sensitive person maybe I need to take a different approach or whatever and and I

also always say to participants um you know at the end of my thing of like if if for any reason these are totally optional so for any reason you want to stop at any time like you can stop to

just be respectful of privacy if I've I have had people who are not in actually prototyping scenario but in other scenarios who feel pretty privacy sensitive and respecting that and just ending the interview if that's what

needs to happen. You know,

that's lovely. Jennifer, thank you so much for that fantastic talk. Give me a round of applause.

You can chill with me as I whatever you want to do. Thank you so much. All

much. All right, so we are uh we're at the end of this day this week. But before we leave, just want to hang with you for maybe a

minute or two uh as I reflect on what's happened this week and what's happened today. At the start of the day, I was

today. At the start of the day, I was chatting with Alec and we're sort of joking around about a couple of things, but something that he said is, you know,

we don't have professional speakers at UXR. We don't bring in Gary Vaynerchuk.

UXR. We don't bring in Gary Vaynerchuk.

We don't bring in put the name there. We

have people here who are real researchers, real research leaders who are sharing from their experiences, the things that they have done, the things

that they've tried, what's worked, what hasn't worked. And what has struck me as

hasn't worked. And what has struck me as I've been sitting over here listening to all these talks is uh it is so cool that we have people who

are willing to just come and talk, right? I mean before I actually go on I

right? I mean before I actually go on I I I should have started with shout out to all of you for being here. Let's give

yourselves a round of applause for actually being here and those of you online because otherwise there would have been eight presenters speaking to

sponsors and eight other presenters and that would have been awkward. But I do think there's something really powerful about being in a community where people

are willing to just come on stage and share what's really happening.

But the thing I want to leave you with as a challenge is when are you going to be up here? When are you going to be the

here? When are you going to be the person who goes, "You know what? I'm a

normal person like Jennifer, like Reggie. I'm a normal person like

Reggie. I'm a normal person like everyone who's been up here. And I'm

going to take a step to not be mediocre and really push myself to advance our community and our practice.

Anyone of you sitting here or anyone of you watching has the ability to be on this stage or not even this stage to be a person who is

building into the lives into the work of other UXRs because we really only advance as a community. We really only

build up what we're doing by trying things, experimenting on our own, sharing that, seeing what comes out. And

so my challenge to you as we wrap up ux.com 2025 and we look forward to ux.com 2026 2026 people. What is going on?

Next year is going to be 2026. But my

challenge to you is not only to do the things that you've been challenged to do during the sessions, but to think

about not being mediocre, advancing what you're doing in your practice, but also share it. Don't keep the stuff to yourself.

it. Don't keep the stuff to yourself.

Don't keep it just to your team. We

don't get better unless each of us help each other out. So do the things that you've been encouraged to do today and this week, but please, please, please,

every one of us, let's share what we're doing to help this community advance.

And with that, I thank you for your time. I thank the team who has made this

time. I thank the team who has made this happen, the sponsors. It has been a blast.

[Applause] Stick around. Hang out with some people

Stick around. Hang out with some people who you've just met or people you've known for a long time. Grab a beer. Go

out for dinner. Happy 2025.

dinner. Happy 2025.

[Applause] [Music]

Hey hey hey hey hey hey hey hey hey hey hey.

I don't feel [Music]