Facebook/Meta Product Manager Interviews: Execution TRADEOFF Questions

one of the toughest type of pm interview questions are trade-off questions like how would you decide between launching x versus why in a previous video

i came up with a framework to help you stand out in these type of questions and so many of you asked for more examples and you know i always deliver so today

we're gonna go through another example question hey guys i'm diana and i'm a senior product manager at a tech company in silicon valley and i bring you the best

tips to get into product management and teach you how to succeed once you've made it so the framework i'm going to go through today is an even updated version

from the one i shared last time it's a six part step starting with first the mission the goal and the hypothesis of the feature we're talking about

and here you want to talk about how the goal of the product is interconnected with the mission of the company and you also want to share the hypothesis of why you think we're

considering between these hypotheses in helping the product accomplish its goal second you want to share some success metrics

defining this product and some of these will be used in your analysis for the a b test third you want to talk about the hypothesized

impact that x versus y is going to have on your metrics number four you want to talk about other options

that you might consider testing in the a b test number five then you want to design your a b test and number six you want to analyze the

metrics to help you come up with a ship no ship retest decision if you want an in-depth explanation of this framework take a look at the last video where i

went deeper into the framework to explain each section so with that let's get started on today's example how would you decide between showing

instagram stories for 24 hours or 48 hours so let's jump into the framework so first the mission goal and hypothesis

the mission of instagram is to capture and share the world's moments and stories lowers the barrier to sharing the world's moments because stories

lives on the feed temporarily and hence people feel less pressure to post high fidelity content now my hypothesis on why we're considering 48 hours because

today instagram stories lives on the feed for 24 hours is the longer content stays on the platform the more time it has to allow for

someone to engage with it hence this might increase the engagement per story and that might possibly get content creators to actually create more

content if they feel like their content is light and because stories live on the platform for longer there's more supply in

stories which might cause people to spend more time on instagram stories next we're going to go into metrics so what are some top line metrics we can

think that are important for stories my number one metric is the number of stories that have at least one engagement per week and for stories

an engagement counts as a reaction or a dm and this metric is important because it represents the connection that happens from the content this is the

intersection when content creators and consumers intersect and connect another one of my top line metrics for stories is the average time spent on

stories per weekly active user and that's to represent that people are spending a good chunk of their time on stories when we think about metrics for stories

we want to think about both sides of the ecosystem which means content creators and consumers one of the first creator side metrics i want to measure

is the total number of weekly active craters on stories per week over the number of weekly active users on stories

because that represents the participation rate of creators on the platform and we might want to see if this will go up given the 48 hours

versus 24 hours a second metric on the creation side is the average number of stories posted per week because in order to grow the supply side

we can either increase the number of people posting stories or the number of stories each of them post and again this might go up if content

creators feel like their stories are getting a ton of engagement now on the consumer side what would you measure again if i wanted to increase

my top line goal which was the number of stories that have at least one engagement the levers that i could pull and i'm thinking about the funnel is first

the number of stories impressions which tells me how many stories people are seeing available on their home screen so imagine i can actually make

the stories each take up less space so people would see that there's more story supply and that might lead to more clicks on stories another metric is a number of stories

that are clicked on or you might even measure the click-through rate to tell you how relevant the stories seem to the user the third thing under

consumer metrics is a number of stories watched and i might define watched as something seen for more than three seconds and then even further down the funnel is

a number of stories that have at least one engagement which i mentioned is our top line metric already now we also want to check whether this new considerate feature of

having instagram stories for 48 hours is going to have a good impact on the instagram ecosystem overall so some things i might want to measure

to learn if that's the case is count the average time spent per weekly active user and i might want to measure the number of weekly active users on the platform have you learned something new

so far if so like the video and subscribe to the channel because next week we're gonna have even more videos telling you

how to ace the product manager interview now let's dive into our third step which is the hypothesized impact on these

metrics first i want to narrow down the most important metrics that are going to impact my decision on a ship no ship for this new feature

we're considering so the four would be number one or top line metric which is the number of stories that have at least one engagement per week secondly is the

average time spent on stories per weekly active user thirdly i also want to see the average time spent per weekly active instagram user is going up

and fourth i want to measure the number of weekly active users now let's jump into the hypothesized impact on some of the metrics we talked about so i'm only

going to do this for the key metrics that i think we're going to move with 48 hours versus 24 hours so the first one being the number of stories with at

least one engagement and i hypothesize that this metric is going to go up because the longer content stays on the platform the more opportunity it has to get engagement

number three i think average time spent per weekly active story user is also going to go up because again more supply and more content on the platform means

there'll be more stories for people to view and engage with and this might lead to an increase in my fourth metric which is the average time spent on

instagram per weekly active user because if they're spending more time on stories per week then as long as that doesn't cannibalize

other features they're using time spent overall for instagram would go up we could also see an increase in weekly active users on stories

now let's talk about variations that i might consider testing beyond the 24 versus 48 hours i may consider testing another option

where users can actually choose between 24 versus 48 hours versus us choosing for them one versus the other and this option might still default to

48 hours but allows users to choose 24 hours if they prefer that and this might be important to help us cater for those power users

who are already posting frequently and they don't want their content to stay on the platform for too long because they want to show people fresh content and we want to make sure we keep these power

users because they drive a huge percentage of the content so with that let's go into designing our a b test

it's important to test our hypotheses and an a b test is a great way to do that and a b test can help us measure the incremental impact and use data to

decide what to ship versus not so in this case the control would be the 24 hours which is what stories is today option number one

is to show stories for 48 hours and option two is the default stories two four to eight hours but give users the option to choose 24 hours and how i

would make sure this a b test is successful is i would work with our data scientists and our engineers to first make sure we log all the metrics for unbiased results

i want to make sure there's enough randomization and for statistical significance we'll want to make sure there's enough people in the test to power the test

and i would ask my data scientists to do a power calculation beforehand to make sure we have enough people in the test and run it for a long enough time to

make sure it's that sig i'd monitor the results and wait to see the impact level off before i make a conclusive analysis

then sure my test isn't being impacted by novelty effects okay lastly i want to go through the different scenarios where i would ship no ship or retest so let's

say we get the results and which options should be shipped i would look for the option where we're seeing the highest incremental impact on the key metrics that identified and make sure there's no

major regressions on other metrics in a no ship decision that would mean my test options actually show a negative impact on my key metrics

say for some reason the 48 hour option actually led to less engagement per story because maybe people are overwhelmed by the number of stories i

would also not ship any of the options if the test options led to a regression on some really important metric maybe something like ads views or if it

significantly cannibalized time spent on instagram feed i would re-run the test if we didn't get statistically significant results

or if we got neutral results on some of the key metrics but saw that for the other metrics they were directionally positive okay guys and that's how i would end a

trade-off question so again here's a summary of the framework that i use to answer these type of questions and we just went through an example question some bonus

tips for you is to prioritize the key metrics that are going to impact your decision and the second tip is to show that you

can design an a b test without it being biased or not statistically significant so we just ran through another example of a trade-off question if you're

wondering how to answer questions like how do i set goals for x product or what would you do if you saw why percent decrease in this metric take

a look at these two other videos that tackle this type of question and if you want to make sure you don't miss any of these videos to help you ace the product manager interview you'll

want to subscribe to the channel and make sure to like this video thanks guys