Check the Event Details

Off to the topic at hand,

I was going to start writing about Product Sense after covering market sizing and guesstimates in the last few posts (here, here and here). But analytical seems like a topic that’s a natural extension of market sizing and guesstimates.

Analytical skills aren’t just important for PMs. They are also quite important in adjacent roles — business or product analyst, data scientists, consulting, program management, etc. Every one of these people have to work with data to figure out insights.

But the burden of using data to create insights, and translating that insight into the impact lies with product teams. To do that, product team needs to have a deeper understanding of:

1. Defining the metrics

   1. Product dashboards, and defining various levels of metrics in the dashboards

   2. NSMs (north star metrics)

   3. Success metrics for a product launch, and guardrail metrics

2. Merging user research and data merge to create powerful insights

3. Root cause analysis of a problem using data, like a metric going down (or up)

4. Creating a plan to move a metric

The above skills are in the increasing order of complexity. For example, item #4 (moving a metric) is something that is the hardest to do among all the others.

While PMs require all the skills listed above on the job, when it comes to the interviews, it’s hard to display #2 (merging research and data to create insights). So those questions don’t appear in the interviews. Further, the type of skill tested also changes with levels. At an APM/PM level, you would get more questions around defining metrics (success metrics, NSM, dashboards) and RCA, whereas at an SPM/GPM level, you start getting problems of creating a product strategy to move a metric.

We are going to cover these items in the next set of essays. But one of the more interesting things that has happened in the past 1.5 years is large language models (LLMs). And one of the notions I had around analytics is —

Analytics is also lot more structured and objective than product sense or strategy interviews, the response of LLMs might be pretty effective here just as it happens in coding.

In other words, can we start using these LLMs as a product companion when it comes to analytics? Since ChatGPT is available at your service 24x7, it’s an amazing companion to have. If the answer to the first question is yes, to what extent should we be using them?

Let’s kickstart this post with the first item: Defining the product metrics!

Defining the Product Metrics with LLMs

Defining the product metrics is the core responsibility for a PM at every level. And it appears quite frequently in the PM Interviews. Here are some sample questions —

• If you were the PM for Lyft, what dashboard would you build to track health of the app? (Google)

• Imagine you're a PM for Netflix. Your team launched Netflix Podcasts 6 months ago. How do you measure success? (Meta)

• What should Airbnb's key north star metrics be? (Airbnb, Google)

Let’s start with the simplest problem for the LLMs — building a product dashboard.

Here is the prompt —

Imagine that you get this question in the interview as a PM candidate. How will you answer the following question?

”You are the PM of the food marketplace product - Food Delivery App like Zomato or DoorDash. You are tasked to create a product dashboard that can be used by product team to track and monitor the important metrics and trends of the product. How would you do that?”

Here is the response from ChatGPT-4. You should surf through the response below and evaluate positives and negatives for yourself before moving on:

As the Product Manager for Food Delivery, my approach to creating a product dashboard involves several key steps to ensure it effectively tracks and monitors the most important metrics and trends. Here’s how I would go about it:

1. Understanding the Business and Product Goals

First and foremost, it’s crucial to align the dashboard with the overarching business and product goals. These typically include:

• User Acquisition and Growth: Attracting new users to the platform.

• User Retention: Ensuring users continue to engage with the app.

• Revenue Growth: Increasing overall revenue through more orders and higher order values.

• Operational Efficiency: Streamlining delivery operations to ensure timely and cost-effective deliveries.

• Customer Satisfaction: Providing excellent service to keep customers happy and loyal.

2. Identifying Key Stakeholders and Their Needs

I would work closely with various stakeholders to understand their specific needs and how the dashboard can support their roles:

• Product Managers: Need insights into user engagement, feature usage, and retention rates.

• Marketing Team: Focused on user acquisition metrics and the effectiveness of marketing campaigns.

• Operations Team: Interested in delivery efficiency, partner performance, and logistics.

• Finance Team: Requires detailed revenue, cost, and profitability metrics.

• Customer Support Team: Needs data on support tickets, resolution times, and customer satisfaction.

3. Defining Key Metrics

Based on the goals and stakeholder needs, I would define key metrics for the dashboard:

User Metrics

• Daily Active Users (DAU) / Monthly Active Users (MAU): To track engagement.

• User Retention Rate: To understand how well we retain users over time.

• New User Sign-ups: To monitor user acquisition.

Order Metrics

• Total Orders: To gauge platform usage.

• Average Order Value (AOV): To measure revenue per order.

• Order Frequency: To track how often users place orders.

Delivery Metrics

• Average Delivery Time: To ensure timely deliveries.

• On-time Delivery Rate: To measure delivery reliability.

• Delivery Partner Efficiency: To track orders per delivery partner and distance traveled.

Operational Metrics

• Restaurant Performance: Including order acceptance rates, preparation times, and user ratings.

• Cancellation Rate: To understand why orders are canceled.

• Customer Support Metrics: Number of support tickets, resolution times, and satisfaction ratings.

Financial Metrics

• Revenue Metrics: Gross merchandise value (GMV), net revenue, and profit margins.

• Cost Metrics: Delivery costs, customer acquisition costs, and operational expenses.

4. Designing the Dashboard

I would design the dashboard with the following sections:

• Overview Section: Key Performance Indicators (KPIs) for quick insights.

• User Engagement Section: Graphs and charts for DAU/MAU, retention rates, and new user sign-ups.

• Order Analytics Section: Visualizations for total orders, order value, and order frequency.

• Delivery Performance Section: Metrics for average delivery time and on-time delivery rate.

• Operational Efficiency Section: Restaurant performance and cancellation analysis.

• Financial Performance Section: Revenue and cost metrics.

5. Tools and Technologies

To build the dashboard, I would leverage:

• Data Integration Tools: For gathering data from various sources.

• Visualization Tools: Such as Tableau, Power BI, or Looker for creating interactive dashboards.

• Real-Time Monitoring: Ensuring the dashboard reflects the most current information.

• User Access Controls: To provide appropriate access to different team members.

6. Implementation Plan

• Requirement Gathering: Collaborate with stakeholders to define metrics and dashboard requirements.

• Data Collection: Identify data sources and establish data pipelines.

• Dashboard Design and Development: Build and test the dashboard for accuracy and reliability.

• Launch and Training: Deploy the dashboard and train the product team on its usage.

• Continuous Improvement: Regularly update the dashboard based on feedback and changing business needs.

Summary

By aligning the dashboard with business goals, understanding stakeholder needs, defining key metrics, and using the right tools and technologies, I can create a comprehensive and actionable product dashboard. This approach ensures the dashboard provides valuable insights that drive informed decision-making and contribute to the success of Food Delivery App.

I am happy with the answer that ChatGPT provides. The positives are:

1. It is well-structured, and covers all the important stages of building a product dashboard. For example, it talks about items in planning stage as well as implementation stage.

2. It covers all 3 sides of the marketplaces — users, delivery partners, restaurants. So the dashboard output covers all the important aspects.

The negatives are:

1. It misses the fact highlighted in the question that the dashboard will be used by the product team, and not for all teams.

2. As a result, it doesn’t grasp the finer details of a product experience like user activation rate, search versus recommendation behaviour, feature usage, product performance, etc.

I went ahead with asking additional follow-up questions to refine the response, and based on what I saw, here is what I would recommend when it comes to using ChatGPT.

Using ChatGPT as Companion

There are two key issues with using ChatGPT as a companion.

The biggest issue lies in the fact that if you don’t know what a right answer looks like, ChatGPT may push you down a wrong path. For example, if someone misses that the Zomato interview question had Product Teams as an end user for the dashboard, they would likely go ahead with the first answer provided by ChatGPT and miss on the metrics relevant for the answer.

The second issue is that sometimes these LLMs will hallucinate, especially when you ask them a question belonging to a niche product. Since we don’t know when and where a particular LLM will hallucinate, the ability to separate right from wrong answer becomes even more important in this case.

And because of these two issues, I won’t recommend using ChatGPT on an unknown problem, especially the unknown problems on the job. Making a mistake in an interview might be acceptable, but doing so on the job can have disastrous effects.

The best use of ChatGPT as a companion is to practice and finding your blindspots in the interview prep. It can be used as a tool once you have gone through frameworks and some good practice problems elsewhere. That way, you aren’t going in with many unknowns.

The last caution is that it’s hard to get to a quite nuanced answer from ChatGPT. These nuanced answers are pretty important for the PM job, and product interviews at senior levels.

But should we just rely on a sample answer to conclude all this? The conclusions feel like a stretch given that we have just taken a problems and seen some answers.

That’s a good question. To fully appreciate and get convinced about the recommendations, you need to understand a thought experiment known as the Chinese Room Argument.

The Chinese Room Argument

Alan Turing, the genius often considered the father of modern computer science, proposed Turing test in 1950. The test proposes that if a computer could answer questions posed by a remote human interrogator in such a way that the interrogator could not distinguish the computer’s answers from those of a human subject, then the computer could be said to be intelligent and to think.

The Chinese Room Argument was a thought experiment proposed by John Searle in 1980s, and it came after the validity of Turing Test.

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program).

Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input).

And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output).

The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have.

— Source: https://plato.stanford.edu/entries/chinese-room/

The Chinese Room Argument undermines the validity of the Turing test.

Now there is debate around this thought experiment. There continues to be significant disagreement about what makes a computer similar to humans. The debate around LLMs is similar.

Sam Altman (CEO of OpenAI) believe that LLMs would lead to artificial general intelligence (AGI) because they (ChatGPT) seem like the way humans would answer these questions. Over time, we will get a breakthrough on this path and reach AGI.

For others (like Meta Chief Scientist, Yann Lecun), LLMs are just a more complex set of rules running on much faster hardware. He argues that LLMs are not a sufficient path to AGI because they lack essential capabilities for intelligent beings, such as understanding and reasoning about the physical world.

The debate has been going on for some time.

Just a few days back, Satya Nadella spoke about how we should ‘Stop treating AI Like Humans, and start treating it like a Tool’. Here is a 3-minutes clip where he talks about how AI is more like an assistant than a human-like intelligence.

Satya would be one of the few people to know the limits of LLMs, given that he has invested in Open AI and manages on the largest tech companies across the globe. And he seems to be thinking that a Chinese Room which can behave like a human isn’t akin to human intelligence.

The Verdict

You need to keep Satya’s comment in mind as a PM while using ChatGPT. They definitely improve the productivity of PMs, but as for doing a PM job even in simpler areas like defining metrics, they can do a poor job at times.

If we merge the Chinese Room Argument for LLMs, and the analysis of sample responses we received, we can now see why it’s hard to rely LLMs for generating insights like a human can. They are definitely better than humans in many regards, but so are calculators.

So use them as a productivity tool on the job, but not as the sole source of research and insights.

As for interview companion, it can do a fantastic job if you want some good practice after preparation. With the voice and video capabilities added to ChatGPT 4-o, you can even do mock interviews and analyse your response later.

That would be all for this week! Do join the Saturday event on Product-Market Fit if you can.

Thanks,

Deepak

1. Product Interviews

2. Operating Well as a PM

I have previously written a long series (42 posts long) on Product-led Growth. You can get it in the email when you subscribe, or can find it on the newsletter home page.

10,000+ smart, curious folks have subscribed to the Growth Catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇

Subscribe now

Let’s dive in the topic now!

When I started pmcurve in early 2022, I was very clear what I didn’t want to do. I didn’t want to create yet another course for ‘Cracking the PM Interview’. From a purely financial perspective, it did look like a good market. But a lot of good and ambitious folks have already done it, and there wasn’t much unique value to add there in my opinion.

So I went after a problem that I wanted to solve for my younger self — creating programs that help you master PM domain, and build+grow successful products.

15 months, 400+ paid members, and 2 unique programs on Advanced Growth and Product Sense later, I am not so sure of ‘not doing anything around product interviews’. Writing has a unique way to build or discard conviction around things we aren’t sure about. And that’s the reason I have started writing the Mastering the PM Interview series as part of this newsletter.

So you may ask the natural question — what changed? The experience of building pmcurve played an important role in arriving at the thesis of this essay: The Parallelism of Product Jobs and Interviews.

We tend to believe that product interviews are broken and far from what the real job demands. I used to believe the same, and that’s why the goal of pmcurve (helping PMs be better at their jobs) didn’t align with interviews.

The problem is that since we treat interviews and jobs as different, we don’t make best use of the things that make us a good PM in the interviews. We also don’t see interview prep as a way to fill our gaps. We do just enough prep to hack the interviews, which is a suboptimal use of that time.

So I entertained the idea that started emerging while designing these programs for experienced PMs — what if we started drawing parallels between the job and interviews? It could create a powerful effect in mastering the PM domain.

But to understand the parallelism, it’s important to understand how I arrived there.

The Genesis of pmcurve

To understand how I arrived at the parallelism, you need to understand the thesis pmcurve started with. I have a strong feeling that without understanding the ‘Why’ behind pmcurve, you won’t be able to fully grasp the current thesis of parallelism.

“The Big Picture— The needs and preferences of Indian consumer is quite different than developed economies like US, Europe, and China. And while India is a large economy, the Indian market is an aggregation of many sub-markets. Each sub-market has its own needs and preference. All of this makes it way more difficult to scale a product because you have to solve for all these sub-markets. Otherwise, you will hit the plateau of growth.

So to scale a product, PMs need to continuously solve tons of problems. And that requires strong PMs in the ecosystem along with strong founders. But why can’t just strong founders be enough?

Significance of the PM role — Besides founders, PM is the highest leverage role in a tech startup because it sits at the intersection of tech, design, and business. The most expensive resources hired by the company (design, engineering) are actually working on decisions taken by a PM. A strong PM is the difference between success/failure in a company because the founder/CEO needs PMs to be their eyes and ears when it comes to ever-changing customer needs.

Problem — The key problem in the current market is that most PMs don’t have way to train themselves to be their best version. There is enough resources on how to crack the interviews, but there is no way to know how good you are in a certain area, and where you need help once you are in the role. Further, we lack India-specific PM case studies, content, and community.

Goal of pmcurve — From the Day 1, the goal of the pmcurve would be to help experienced PMs building for India create impact in their jobs via an affordable learning solution. This visible impact that they create on jobs will certainly fast-track their career.

Why is now the right time for pmcurve? — Indian tech ecosystem has gone through couple of cycles (ups in 2014-16, 2021-22). This does couple of important things. First, we have enough specific case studies to create a different curriculum for India. Second, we also have observed the problems experienced PMs face around mentorship/learning for some time now, and know that these aren’t going away on their own.

USP of pmcurve — Application Based, Experienced PM-focussed, Affordable, Content and Community that helps PMs in India

The Last 15 Months

The early GTM strategy of pmcurve was to introduce programs that helped PMs in areas they struggled the most. Based on my own experience and interviews, there were two areas where experienced PMs had most struggle:

1. Growing a product: Most PMs know how to build and launch a product. But when it comes to growing the users or revenue, they struggle with it. Reforge runs some reputed global programs around Growth for PMs.

2. Making the right product decisions (tactical and strategic): Product sense is usually the term we use to talk about making the right product decisions in day-to-day work of a PM. Product strategy is the term used when we talk about medium-to-long term strategic decisions. While there are few courses around product strategy across the globe, there is no course/program that teaches product sense.

Based on the insight, So we launched two big programs —

1. ‘Advanced Growth Program’ as the first program that helps PMs understand growth problems, build growth models, and come up with ways to move levers to attain growth. We added 600+ objective problems in this program, which is pretty unique. The reason to add these objective problems is to ensure PMs remember and apply what they learn.

2. ‘Product Sense and Strategy’ as the second program. I clubbed both sense and strategy since I believe that product sense and strategy aren’t separate. They are closely related. This took a lot of my energy and time in research because there isn’t any curriculum for product sense globally.

As a part of building these programs, we had to create and deliver the right content, case studies, and assessments. We also had to get the right PMs into the community so that the discussion don’t dilute over time. Till 4 months back, I was very sure that I didn’t want to address anything around the PM Interview. That changed when I started teaching the first cohort of Product Sense and Strategy in March.

The Change in Thesis

It was while teaching the Product Sense and Strategy program that my thesis started to change.

While designing the curriculum and content, I started seeing parallels between skills expected in product sense rounds for experienced PMs, and skills expected on the job.

I could see few examples from my own career on how interviews have helped me become better at product management, and vice-versa:

1. Interview feeding into the job performance: My interview in 2018 at Google led me to the realisation that I lacked technical knowledge to be a PM who could manage deep-tech products. What followed was a grind to understand tech, which eventually translated into the book ‘Tech Simplified for PMs and Entrepreneurs’. Over the years that followed, I have managed platform and deep-tech (AI) products at companies such as Unacademy, Flipkart — and that did make me a much better PM.

2. Job performance feeding into the interviews:My role at MX Player as Head of Product Growth helped me understand media and entertainment industry very well. To launch new initiatives such as games at MX Player, I had to build deep understanding of the video and games domain. The domain understanding helped me answer one of the key questions in the interviews the next year 'Should Flipkart launch Videos?'. What’s better, my expertise at building a 0 to 1 games platform helped me land the role of head of product - games and gamification at Flipkart.

My next hurdle was creating a map between jobs and interviews, aka the Parallelism.

The Parallelism

If you have a long-enough career in product, you will see the parallel phenomena pretty easily.

To observe it, let’s start at various responsibilities of a PM. PMs are responsible for these in their real-jobs:

1. Roadmapping for the next quarter or year

2. Creating PRDs of new products and new features

3. Doing RCA of a technical problem. Example — App crashes increased yesterday. Why did it happen?

4. Figuring out a growth problem. Example — The transactions are down last week, and have been dipping since last 2 months. Figure out why that’s happening)

5. Solving a growth problem. Example — Come up with the strategy to improve number of transactions on the platform, and get a buy in from leadership)

6. Working with teams well. Example — There has been a conflict with engineering on timelines. How do we move forward?

If we look at the interview problems, they mirror these interview problems:

1. Roadmapping is real-life response to ‘How would you launch X?’ — Roadmapping requires listing customer problems, looking at solutions, prioritising the features, etc. You are supposed to display the same skills in a limited time in the problems such as ‘How would you launch a new travel website for elderly people?’

2. PRDs are the real-life response to ‘How do you design or improve a product X?’ — Product Sense Interviews mirror PRDs. They start with the problem, why it’s important for the company, customers, their needs, solutions, which solution needs to be prioritised, success metrics, initial GTM, pricing, and launch strategy.

3. Execution interviews nowadays have become common for companies such as Meta and Google. These interviews have questions that mirror problems in real PM jobs. For example, in the real PM job, we often face these technical or growth issues like broken funnels, DAU going down, crashes increased, etc. We first have to understand the root cause of these problems and then have to discuss with different teams to solve it.

   Execution interviews gauge candidate’s ability to systematically understand the issues and solve them. Usually, you get better at real job problem-solving as you get better in the execution interviews, and vice-versa.

4. At leadership product interviews, the discussion also spans around product strategy problems. It could be entering into a new market, or building an entirely new product. This is very similar to boardroom discussions at highest levels which talks about company’s focus into new areas.

5. Behavioural questions in interviews tests how you handle the stressors at work, and your operating philosophy. Such questions are all about seeing whether you will be able to work well with the existing teams and thrive at work.

Interviews and Jobs as Two Sides of the Same Coin

The benefit of understanding the parallelism of interviews and jobs is that you can start becoming more conscious when solving a problem at work. You will further seek frameworks, case studies, and structured ways to solve these messy problems.

Being conscious improves the effectiveness in even complex jobs such as surgeon, pilots, etc. as the book The Checklist Manifesto covers. It has definitely helped me improve outcomes at job in the past, and still does at product development initiatives we are working on at the pmcurve.

All of this conscious effort makes you stand out in the interviews because it makes you better at structuring thoughts, seeking the right case studies for a particular context, and storytelling.

Similarly, treating your interview prep as a learning exercise about various different fields and filling your gaps in certain areas (like growth, data, product sense, strategy, etc) is more effective than merely hacking the interviews because it improves your chances of becoming better at job once you convert. It also helps you assess right opportunities in terms of behavioural fit.

This became quite clear when I was teaching product sense over the last couple of months. But that leaves couple of important questions people have asked me around this, and I feel it’s important to cover them:

Is it possible to still be good at the job and bad at interviews? Yes, if you treat them as separate and not prepare well, you won’t be able to transfer the abilities on your job to interviews. If you sit in interviews and solve problems naturally as you solve them, you would do a fab job.

Is it possible to still be good at the interviews and bad at the job? Yes, Interviews can’t test for many things. So it’s possible to hack interviews while still not able to do an excellent job.

My advise would be seeing the parallelism so that you are always prepared for interviews, and always doing some great work :)

What’s Next?

While the parallelism is obvious now, someone needs to layout how to use this framework. In the coming weeks, I am working on a proposal for Mastering Experienced PM Interviews. Goes without saying, it would be quite affordable :)

Stay tuned, and have a good day!

Deepak

Before we start, I'd like to make a quick announcement!

The applications for the next live cohort of the Advanced Growth Program are open now. It's a unique learning experience via custom learning software and live classes. About 300+ experienced PMs have gone through the program and loved it.

Check the Program

In 1980, McKinsey & Company was commissioned by AT&T to forecast cell phone penetration in the U.S. by 2000. AT&T’s Bell Labs had invented cellular telephony. The McKinsey’s prediction was 900,000 subscribers. Based on this prediction, AT&T decided there was not much future to the cellphones. And how off was the prediction? It was > 100x off from the actual figure, 109 Million.

A decade later in 1993, to rejoin the cellular market, AT&T had to acquire McCaw Cellular for $12.6 Billion. By 2011, the number of cell phone subscribers worldwide had surpassed 5 Billion.

Market Sizing is a Tricky Business

Forecasting market size is tricky because we have little idea around how the future will pan out. Hundreds of pieces from reputed sources have been written about why predicting the future is hard to predict. In a way, we have made our peace with it. And I hope that after reading this piece, you would be little skeptical to such market predictions about future.

But what about sizing the current market? While it looks an easy thing to do, we will see examples in the post that it’s harder to size the market correctly in the case of an innovation.

PMs are in the business of innovation. And that’s why it becomes important for them to exercise this muscle. A good estimation of market for a new product line could be the difference between a red light and a green light within the organisation. Years down the line, a good estimation could also be the difference between a successful product and a failed one.

And with that, let’s start!

Types of Market Sizing Problems

Market sizing problems can be of two types:

1. Market sizing for incumbent products : This is what traditional case questions look like. The questions is about an existing market. An example of this is — How many fridges are sold every year in India?​

2. Market sizing for innovation products: This is when you introduce a product in the market that doesn’t exist. An example of this is — Google has built a teleportation device which can instantaneously moving things from one location to another. Estimate the market size of this device.

The innovation product market sizing are quite relevant for PMs in their jobs, and like we mentioned earlier, it’s also the harder thing to do. The good news is that the same framework can be applied to both types of market sizing. Next, let’s cover the framework to do the market sizing that’s applicable to both types of market sizing.

Approach for Market Sizing Problems

The approach for incumbent as well as innovation products is similar, if not the same. Here are the steps you can take:

Step 1: Clarify what the product is — If you can’t describe the product offering well, you wouldn’t be able to size the market. For example, in the teleportation device market sizing, you have to understand it’s capabilities and limitations.

• What’s the maximum distance it can teleport?

• Does it teleport living as well as non-living things?

• What’s the weight and volume limit?

• Does it need a particular tunnel like Hyperloop, or can it teleport through air/water/solid/vacuum?

Depending on the answer to the questions above, the customer problems that the product solves will change, and the final market size will change. As an example, if it can only transport things with weight <10 kg, the market gets significantly limited as compared to when it can transport up to 100 kg.

Step 2: Clarify who the end user is — The second step is the discussion about the user of the product. Who exactly is the user? Identify the consumer segments for whom the product solves the problems.

Does it serve businesses (B2B) or consumers (B2C)?

In the B2B market, the segmentation can be further done in terms of size of the company, type of the business, annual revenue, etc.

In case of a B2C market, the segmentation can be demography based or behavioural.

1. Demographics: On the basis of age, gender, income

2. Behavioural: On the basis of their behaviour and attitude

There are two important things to remember in segmentation:

First, while the segmentation can be done in multiple way, you should choose one that’s relevant to the product. For example, while estimating the new fridges sold in India, the age based segmentation isn’t of much value. We have to look at income levels of households and estimate starting from those who can afford a fridge.

Similarly, while segmenting users for a dating app, we have to segment them at a behavioural level. There could be segment interested in casual dating, whereas others looking for serious partners.

Second, while doing the segmentation, it is also important to take the MECE segments (mutually exclusive, collectively exhaustive). In absence of MECE, you will end up double or triple counting few numbers. Age based or gender based segmentation is MECE as one consumer will fall in only one segment. On the contrary, if we do a segmentation based on people buying premium vs economic brands, we might be double counting people who fall in both categories.

While you can always ask the interviewer on who the user is, PM interviewers usually ask the candidate to pick the relevant market segments based on the problem the product solves. In such cases, the interviewers wants to check your judgement. This is similar to PM job where you won’t get this clarification from someone else, but will have to pick it for yourself and justify. So don’t ask too many questions if they ask you to make an assumption yourself. Pick the segment and explain why you picked the segment.

Step 3: Clarify other variables like time and location — This is the last step of clarification phase :)

It is important to ask the geography and timeframe within which we want to evaluate the market size. A good advice is to pick a market you are already familiar with, especially when it comes to age distribution, digital penetration, and income distribution. You can go through the last post to get those data for India and US market if you haven’t done so already.

It is also good to check whether we are looking to get the market size in terms of volume (number of users/ units), or value (revenue).

Step 4: Identify the channels (Optional) — Once we have clarity around the product and the consumer, we have to ask — how will the product reach the customers? This step is optional because if it’s a purely digital product like a social media or messaging app, you know that the channel to acquire customers will be purely online. But if it’s a physical product, you need to list the channels using which the product is sold. Is it online vs offline vs omni-channel? The answer will change the market segment apt for the product.

For example, new fridges sold online exclusively is a different market than new fridges sold offline. It’s important to clarify this while solving the problem.

Step 5: Pick the approach for estimation — This step is the most crucial for the estimation. Usually, there are two decisions you need to make while picking the approach:

1. Should you pick demand or supply side estimation?

   Imagine that we are talking about the new fridges sold in India.

   1. Do we want to pick the number of suppliers and multiply it by # of fridges sold by each supplier?

   2. Or do we want to take demand side of the equation i.e. number of fridges in the country / lifespan of a fridge?

   A general rule of thumb is to pick the side where you are able to obtain the variables by general observation of the world. In the fridge example, it’s hard for you to get total number of suppliers and average number of fridge sold by supplier through general observation. It’s easier to take demand side approach where we take the # of households that can afford a fridge and calculate from there. In certain cases, supply side estimations can work. But most of the times, demand side estimations work in interviews.

   You should note that it’s not that supply side estimations aren’t useful. Supply side estimations are more useful at the real PM job where you have data available for major players in a market, and their market share. For example, in markets where there are a few large players, supply-side estimation based on these player’s revenue can provide a fairly accurate measure of the market size. It works in real life because sales figures are often available through company reports, industry reports, or regulatory filings. These figures can’t be fetched in the real interviews from an intelligent guess.

2. Should you pick a top-down versus bottoms-up approach? The answer to this question also ties to — take the approach where you can make intelligent guesses through general observation.

   For example, in the case of new fridges sold every year in India, you can start at the top by looking at number of households, and income distribution of households. But imagine that you have to estimate the # of shampoo bottles sold. You have to start with the estimating the consumption per person, and then move upwards.

The next two steps are easy and not different from what we covered in part 1.

Step 6: Draw the issue tree and calculate. Pick easy numbers that are justifiable.

Step 7: Perform sanity check

I believe doing an estimation for an incumbent product should be fairly easy once you follow these steps, so let’s focus on an innovation product market sizing.

Calculating Market Size for Teleportation Device

Let’s apply the above framework in teleportation device market sizing for Google.

Step 1: Clarify what the product is : In the case of teleportation device, here is the summary you got from the interviewer

• What’s the maximum distance it can teleport? 5 km

• Does it teleport living as well as non-living things? both

• What’s the weight and volume limit? up to 150 kg in one go

• Does it need a particular tunnel like hyperloop, or can it teleport through air/water/solid/vacuum? It can teleport through air/water/solid/vacuum

As we understand the capabilities of the product, it becomes clear that it can be used for short distance transportation, but it can’t be used as a replacement for inter-city or inter-country travel. But just to make sure we are on the right track, we can clarify it.

• While there is a possibility that we can create long-distance transportation by connecting a series of such devices. Like 100 devices can be connected in a row taking the person to 500 km. The experience might be problematic with the person reappearing at each device and then moving on to next. So I am going to ignore the long-distance travel use-case. Is that okay? Yes, that’s okay.

At this point, you can also ask the interviewer to describe the product's share and how it works.

Interviewer: It’s kinda like a telephone booth. You enter into it, key in the coordinates, swipe the card, and move to the next location. Pretty similar to how it happens in Matrix.

Step 2: Clarify who the user is — In this step, you can ask the interviewer on whether we have already decided on the market or it’s an open question. Suppose it’s an open question, we have to pick the relevant users. We already understand what problem the device solves — short distance travel of humans.

What else are we missing before we narrow down to the target segments? Pricing!

At this point, we can ask for pricing parameters of the product. The answer from the interviewer can be as straightforward as $5/km, or the interviewer can ask you to set the pricing. To set the pricing, we need to go deeper into how to price a product, and that’s not very relevant to current market estimate. So we will skip that for now, and assume that we have the pricing set at $5/km of travel.

For $5/km, the cost of an average ride would come to $25 for 5 km. Who would be willing to pay this price?

In the Indian market, we pay anywhere between $5-7 for a ride. It’s too expensive for everyday travel in India. What about the US market? If you are sitting in India and estimating for US, estimate based on the fact that US has a price parity of 3x wrt India (price parity is an indication of how expensive one market is as compared to another market). So the price of a cab ride in US can be $15-20. Because the teleportation device saves so much time, people or their companies would even be willing to pay a premium of 30-50%. So it’s safe to assume that people would be willing to use the teleportation device in the US market. Other markets that are good for this are Europe and China, which are large markets with high capita income.

What about transporting other goods through this device? The metals and other items are usually heavy and per kg travel cost for them would be a lot cheaper than a taxi ride, so let’s not go there for now.

So we have picked everyday travel as the market segment to go after.

Step 3: Clarify the time and geography variables — Based on pricing, we know that we can start with the US market. That said, it’s easier for us to scale in other high capita markets like Europe, China, Canada, etc.

At this point, it would be important to narrow down the geographies further. The reason for that is because the behavior of users vary based on geography when it comes to everyday travel. We know that people in tier-1 cities spend money on taxis. The propensity to spend is much lower in tier-2 and suburbs. People in such areas either use their own transportation, or use public transport. In addition, the volume of everyday travel and trips per square km is usually higher in cities. That’s the reason Uber started in New York and expanded in cities first.

So rather than estimating for the whole US, we can start with the market size for a city only. We can always go back and calculate for tier-2 and above if need be.

Step 4: Identify the channel — we will skip this step as it’s pretty obvious on how customers will reach the product, and vice versa.

Step 5: Pick the approach — We will go for demand side calculation here because we can make a lot of good guesses based on general observation.

• Population of a city > 5 million

• Let’s segment the population

  • Below 5 years (7.5%) — Kids below 5 years of age can be excluded since they stay at home.

  • 5-15 years of age (15%) — School kids travel via buses which are usually cheaper. So we can exclude this segment.

  • 15-20 years of age (7.5%) — College kids who usually need to travel within campus only. So we can exclude this segment.

  • 20-65 years of age (50%) — This is the working population that needs to travel everyday. We have go deeper into this segment.

  • 65+ years of age (20%) — They need to travel occasionally once they retire so we can exclude them as well.

Pro Tip: Note that while you can include a small percentage from retirees and college kids, you need to be careful in doing so. They won’t contribute much to the market size, but can take a good amount of time in interviews.

• Within the 20-75 year of age segment, around 70% is the working population (80% of males are working, and 60% of females are working + 50:50 ratio of population). Rest are unemployed or choose not to work like housewives. That gives us 3.5 million (5 million * 70%) working population.

• An average working person can takes 2 rides a day, 25 days a month. So they would need to spend 50*$20 = $1000 per month on taxi rides. That comes to around $12000 per annum. We can assume that only people earning > $100-150 k would be willing to incur such an expense. Looking at the US distribution of income, this is around 10% of population. In major cities, this number can be higher, say 20% of working population. So the qualified segment becomes 3.5 million * 0.20 = 700 k people

• We have 700 k people spending $12000 per annum. That makes it $8 billion per annum from a city of 5 million people.

• In the US, we can assume that around 30% of population lives in such cities, i.e. 300 million *30% = 90 million. So the market size of US, provided the device is launched in cities, would be 90 million/ 5 million * $8 billion = $ 144 billion

• US economy is large and it’s per capita is pretty high as compared to other large economies such as Chine, India, or Europe. Maybe we can add 20 more cities to such list. So the total market size would come to $300 billion in total.

Step 6: Calculate : we have already done it.

Step 7: Do a sanity check : In this we will need to look at few assumptions:

1. We have taken price per ride as $20-25, and that’s limiting the market to individuals with high salaries. Is it possible to reduce it? If yes, that can significantly increase the market size.

2. We have assumed that all individuals who have the ability to pay will choose such option. In real-life, a lot of people prefer public transport especially in cities where it’s pretty good. Many people prefer the comfort of their cars.

3. Many people would need to walk as compared to getting directly in an Uber. We have assumed these booths are stationary. If they were mobile, the utility could increase because they can be placed in high traffic areas depending on the time of the day. We aren’t sure if that is possibility but that can change the numbers.

4. We have assumed that expansion to other cities across the globe is possible. In reality, there could be regulatory bodies or other rules preventing that.

5. We haven’t included other smaller cities or countries with high capita income. They will also expand the market.

All in all, we can take a lower adoption in qualified segments (say 60%), and also take a cut in the final numbers (70%). So the final number comes around $300 billion *0.6 *0.7 = $130 billion.

If you have gone through the calculations, it would be fair to say that market sizing for an innovation is a lot more tougher than an incumbent. But that’s where PMs add significant value. If you become better at it, you can spot opportunities easily and size them up. In other words, you can build the ability to take your company from X to Y.

Subscribe now

General Guidelines for Estimation Interviews

Here are the general guidelines for the interviews:

1. Think loudly. Share your thought process as you tackle the problem. This keeps the dialogue lively for the interviewer. It also demonstrates your thought process to the interviewer, and they can nudge you if they want you to go in certain direction.

2. Know that it’s fine to pivot your strategy if you discover a more viable direction.

3. Keep your numbers easy for easier calculations. It is perfectly acceptable if you working with rounded figures.

4. Remember the broad numbers for key geographies like US and India.

How to Practice and Be Better at Estimates

Here is what I recommend on how to practice estimates.

Phase 1: Start with practicing by yourself slowly. Think about numbers/stats that are available on the Internet and solve them here. Take as much time as you need. You need to think carefully at first.

Phase 2: Start timing yourself when solving these problems.

Phase 3: Interview with peers

Phase 4: Interview with experts

Phase 5: Get to a real interview

Books around the topic

I only recommend the books I have read personally. So feel free to drop a suggestion in comments if you have found any other terrific book on the topic of market size estimations, and estimations in general.

One book that’s classic around market sizing problems is Case in Point. This book is more relevant for incumbent product market sizing, but it is a good start!

A Word of Caution

Most videos and articles about estimates are filled with brilliant examples, and in many cases the estimates are adjusted after the author looked up the true answers on the Internet. Add to that, there is also a selection bias — people do these estimates and throw out the ones that are way off, or don’t look good.

And that makes it hard to get a realistic picture of how good you are when interviewing. What does it look like in the real interviews when someone isn't “looking” / “cheating”.

I hope you have more realistic expectations of what it's like to do guesstimates when you're just getting started. That will make things pretty exciting when starting out.

Hope you liked it! See you next week,

Deepak

Before we start, I'd like to make a quick announcement!

The applications for the next live cohort of the Advanced Growth Program are open now. It's a unique learning experience via custom learning software and live classes. About 300+ experienced PMs have gone through the program and loved it.

Check the Program

The first atomic bomb exploded at 5:30 AM on July 16, 1945, in the New Mexico desert of the USA. Shortly afterward, Enrico Fermi estimated its explosive yield with a simple calculation while sitting 10 miles away in a camp.

He made the estimate on the spot, using the technique that now bears his name: Fermi estimation. Here is the official account for it.

About 40 seconds after the explosion the air blast reached me. I tried to estimate its strength by dropping from about six feet small pieces of paper before, during, and after the passage of the blast wave. Since, at the time, there was no wind I could observe very distinctly and actually measure the displacement of the pieces of paper that were in the process of falling while the blast was passing. The shift was about 2 1/2 meters, which, at the time, I estimated to correspond to the blast that would be produced by 10,000 tons of TNT.
—Enrico Fermi, Top Secret interview July 16, 1945, declassified in 1965

So, how off was he? The blast's official estimate, which came weeks later, was 21,000 tons of TNT. Fermi's estimate was quite accurate, given that he was sitting 10 miles away in a camp. He was just off by a factor of 2 when we didn't know much about atomic bombs. That's impressive. How did he do it?

The trick in the Fermi’s estimate is to estimate values using only orders of magnitude, a.k.a. powers of ten. For example, if we have to place a bet on the number of cars in India per person, we have to choose in the orders of magnitude, i.e. between 0.1, 1, or 10, 100..etc.

My pick while doing this exercise was 0.10, and the actual number is 0.05 (70 million registered 4-wheelers and above, assume 50-70% of it as number of cars).

But don’t the errors add up and throw out a ridiculous number as an output when you perform complex calculations? It may feel that the final calculation would be far away from the actual number, and yet when we apply it, it actually works.

Let’s try a question built for Fermi estimation: How many petrol pumps are there in Mumbai? You can’t answer “I don’t know” :)

Let’s try Fermi estimation (picking a multiple of 10)

• There are 10 million people in Mumbai (by the rule of “only powers-of-ten,” our pick is 10 million; the other two options are 1 million and 100 million. The actual number in Mumbai happens to be 20 million).

• In Mumbai, there are more vehicles per person owing to the wealth the city accumulates. So between 0.1 and 1, we will pick 1 car per person.

• A vehicle refills one time per month (it’s not 0.1 and not 10).

• A petrol pumps handles 10000 refills per month (the number is not 1000 or 100000. You can do a quick calculation based on our observations in day-to-day life. Each time I have gone to a petrol pump, there are 10 vehicles there getting serviced within the span of 15 mins I am there (not 1 or 100). That leads to 40 cars refilled per hour, and with 12 operational hours per day, we have 480 refills per day. That translates to around 15000 refills per month. Since this number is closer to 10,000, we pick that number).

• With 10 million refills per month (10 million vehicles with 1 refill per month), divided by 10000 refills per petrol pump, we would need 1,000 petrol pumps in Mumbai.

A source says that there are 1162 petrol pumps in Mumbai!

Despite being inaccurate in the process, the end result of 1000 petrol pumps is indeed closer to the actual number of 1162. That’s the power of Fermi Estimation.

If a scientist can use quick, fun estimations in one of the most prolific events in human history, we can too when there is a need on the job!

The Job of a PM and Guesstimates

The job of a product manager is filled with uncertainties, and these uncertainties come in all shapes and sizes.

Imagine that you are a product manager at an e-commerce startup, and in a meeting, the CEO asks for your input on whether the company should go into Women's Workwear. You do a quick Google search, but since it’s a niche market, you don’t get much data around it. You find a couple of sources. The global workwear market is estimated at $32 billion, but you don’t find much data around India. After looking around more, you get another option — to purchase a report around the workwear market at $2900. But with limited info around the topic, you don’t know whether to buy an expensive report and trust it.

So what do you do? You employ your mental faculties to do a quick estimation of what could be a reasonable number so that it’s easy to move forward with a decision. You further have to calculate it from different approaches and triangulate it to a decent number. And that’s what is expected of you as a PM.

Intelligent estimation skills are useful in other places as well. It can help in

• estimating effort and timelines for features on the roadmap, and

• estimating the user or business impact that a feature can have

As you get better at doing these estimations on the job, you also get better at guesstimates, and vice-versa.

Solving Guesstimates in Product Interviews

In the last post, we covered how the real job PM skills actually help in building your advantage in PM interviews. In this post, we talk about how to solve the estimation questions that come frequently in PM interviews.

There is enough literature written about guesstimates, so how is this post going to be different? I wanted this post to become a one-stop guide to guesstimates so that you don't have to read anything else. How do you do that? The simple answer is cover everything. But that is just lazy.

To become a one stop guide, merely covering everything under the sun isn't going to cut it. The framework/ approach has to be so fundamental that everything else can be derived from it. Not to mention, it also has to be clear and easy to understand so that everyone reading it can actually apply it.

We have already answered why companies ask guesstimates by looking at how guesstimates are useful in a PM job.

The guide covers:

1. Companies that love this question

2. Popular interview questions of this type

3. Generally suggested approach or framework to solve such questions

4. An end-to-end example of the approach

5. Handy numbers for solving guesstimates

6. What a better approach to solving such questions look like

7. Key mistakes and examples of such mistakes

8. Test your knowledge of this topic (beginner vs advanced score)

9. Subjective Questions Database for further practice

10. Books, articles and videos around the topic

This post was getting way longer to cover all of it. So, in this post, we cover points 1-5. We would deep dive into 6-10 into the next week’s post. Subscribe to the newsletter so that you don’t miss it!

Subscribe now

Let’s start!

1. Companies that Love Guesstimates

This interview problem is relevant to startups as well as mature companies because, as we mentioned, it is quite relevant to certain skills in the PM job.

Estimation questions are pretty common in product manager interviews at companies like Google, Facebook, and Amazon. Owing to the simplicity of these questions, these questions also get asked in startups.

2. Popular Interview Questions

Here are a few common examples of PM estimation interview questions to provide a better sense. Even if all of these questions are categorised as guesstimates, there are various sub-types that we will cover later and how to approach them. For now, read through them so that you have a good sense of various types of questions:

- What is the market size for driverless cars in 2025?

- What is the storage space required to host all images on Google Street View?

- What is the required Internet bandwidth for an average college campus?

- How many kindergarten teachers are there in the US?

- What is the weight of a school bus?

- Estimate the number of ATMs in India

- Estimate the number of videos on Youtube

- Estimate the market size of self-driving cars

- How many dentists are there in New York?

- How many queries does Gmail get per second?

- How many airplanes are in the air right now in India?

- Estimate the cost of storing Google Photos on the Photos app.

3. Generally Suggested Approach or Framework to Solve This Question with Example

Let’s take an example where you are sitting in a Product interview and the interviewer asks you to estimate the number of YouTube videos watched in India in a single day.

How would you address this question? Below is the recommended method for solving guesstimate problems.

1. Clarify the question

2. Pick an approach

3. Calculate using the approach

4. Sense check your results to make sure your calculations are accurate, and the final number doesn’t look improbable

Let’s apply it to understand this method better!

Clarification

Like any product interview, clarifying questions make sure you don’t make wrong assumptions and are on the wrong track to solve the problem. Here are a few important clarification questions you would need to ask:

• Should we count just the long-form videos or shorts as well?

  • Interviewer: Assume that we have to long form videos

• What should qualify a video as ‘watched’? Is it >1s, >10s, or >30 s?

  • Interviewer: Take >30 s as watched

• There are long ad videos on YouTube as well, which most of the people skip. Should we exclude them?

  • Interviewer: Yes, please exclude them

Approach

There are several approaches you can take to arrive at the number.

1. You can take the total population and multiply it by the average number of videos watched per person per day.

2. You can estimate the total number of videos and multiply it by views per video per day.

3. You can also take the total number of smartphones and smart TVs and multiply it by the average number of videos watched per smart device

The golden rule in picking up an approach is — pick an approach where you can derive the numbers easily when you break it down further.

Let’s critique these approaches. For example, let’s take approach #2. Views per video per day is hard to derive. Further, a total number of YouTube videos is also hard to come by as it is an accumulating number. So, approach #2 isn’t recommended.

You can pick #1 or #3 and arrive at a number.

Here is a step-by-step calculation that you need to do if you pick approach #1:

1. Calculate the number of people with smartphone and smart tv access

2. Calculate the average time spent per user per day on Youtube

3. Calculate the average length of a video watched (this is a function of average length of video and % completion)

You can draw an issue tree to break down these calculations further. Going down to 3 levels in the issue tree covers enough details while not making the guesstimates overly complex.

Calculate

The next step is to put the numbers in the issue tree and back-calculate.

When making these assumptions, it is vital that you pick simple numbers. For instance, you should use 350 days in a year instead of 365. You should assume that the average cost per product is $3.00 instead of $2.50. Rounding numbers will make your calculations easier and decrease your likelihood of making a mistake.

The issue tree above can be updated as below. Note that while making numerical assumptions, we use handy numbers for solving guesstimates. These numbers are shared at the end of this post, so don’t worry if some of these numbers seem too intelligent a guess. After going through the handy numbers section, you would be able to make the same assumptions.

• We take the assumption that 30% of the population can afford a smartphone. This number is one the basis of per capita income in India, where median income is around $100 per month, and top 10% have $300 per month. A person earning more than $100 can afford a smartphone, but as it happen in middle class family, there is just one bread winner (usually husband). And so the # of smartphones per household is 2 in such families. That’s why we take 30% of population owning smartphones — (10% top income) + (next 40% divided by 2)

• Next, we make the assumption that kids population (0-10 years) is around 20% of population because India has young demographics. Further, parents control their consumption, so consumption is lower (30 mins per day). We make assumptions in a similar way as we go through other number. For example, we make an assumptions of 2 hours/ day for retired people because they have lots of time to have fun after retirement :)

• We take average length of video watched as 10 mins because most of the long-form videos are also music, recipes, news snippets, etc. which are shorter than 10 mins. We take average % of video watched before existing as 70% because people leave the videos in between even if they have crossed the >30 s threshold. Please note that this number would have been different if the threshold had been different.

Next, we can calculate from right to left and get

1. 20% of 400 million watches 3/0.7 videos a day

2. 50% of 400 million watches 6/0.7 videos a day

3. 30% of 400 million watches 12/0.7 videos a day

The final number comes out to be 4.1 billion videos per day.

Sanity Check

Most candidates stop talking at the end of step three. They look up and expect their interviewer to tell them if they got to the right result or not. Don’t do that. The best candidates sense-check their results and try to spot their own mistakes before telling their interviewer they are done.

Let’s try to sense check the results.

• We have take 400 million Youtube users in India which doesn’t seem far off.

• The average time spent per user is >1 hour daily. This is a number we need to relook. While the power users of Youtube may spend this much time, not everyone spends 1 hour on Youtube daily. People also spend a lot of their time in listening to music, watching short videos platform (including shorts on Youtube which we have excluded), and on OTT platforms watching shows and movies.

  So we can cut the number by 50% per day per user.

  Further, assuming that Youtube shorts will have 20% of watch time, we can take the final number as 0.8*0.5 = 40%

• We have taken the average length of long-form videos to be 10 mins, which seems right.

So the final number can be 40%* 4.1 billion = 1.6 billion videos a day.

If you are looking to see how off we are from actual numbers, YouTube users view 1 billion hours of video each day. That accounts for around 5 billion YouTube videos being watched each day. India accounts for 10% of Youtube traffic, so by that account, the share of videos watched could be anywhere between 0.5-0.7 billion. So we are off by a factor of 2, but not much.

Thank you Mr. Fermi! :)

5. Handy Numbers for Solving Guesstimates

If you are interviewing for an Indian company, make sure you have handy numbers for guesstimates. Having these numbers handy makes you sound intelligent in interviews, and also provides you a way to anchor and perform sanity check.

If you are interviewing for a global company, make sure to have basic number of US and the world handy, along with India.

Top 10 numbers you should have handy:

1. Total world population = 7.3 billion

2. World smartphone users = 4 billion

3. Population of India = 1.3 billion

4. Urban vs. Rural split in India = 30% urban, 70% rural

5. Households in India = 325 million

6. Smartphone users in India = 600 million

7. Internet penetration in India = 50% = 625 million

8. US population = 330 million

9. Mumbai population = 20 million, NYC population = 8 million

10. Average household size in India = 4, US = 3

Beyond these, age groups and income levels are pretty important as they determine the behaviour of people.

INDIA

1. Age — India has a younger demographic with a median age of 27 years.

   1. 0-15 : 25%

   2. 15-64: 65%

   3. 65+: 10%

2. Income

   

As a rule of thumb, you can remember that top 10% makes INR 25000 per month, or $300 per month. Top 5% makes >$800 per month, and top 3% makes > $1200 per month. That means the number of people making >$1000 per month would be around 3%*1.2 billion = 36 million. This is also, btw, the number of credit card holders in India (4%).

To look at the 600 million smartphone users, we have to look at 50th percentile. The median income is ~ $100 per month.

USA

1. Age is balanced

   1. 0-15 : 17%

   2. 15-64: 65%

   3. 65+: 18%

2. Income distribution

For USA, the median income is around $40 k, whereas top 10% is >$100 k.

You should note at this point that household income distribution is different than individual incomes since multiple people can be earners in a household.

World

Where people live

How much do they earn

What’s Next!

In the next week’s post, we cover how a better approach to solve guesstimates involved identifying sub-types of questions (volumetric, population, consumption, product), and why a different approach should be picked to solve each of them.

Further, we also go into the complex problems such as market sizing and their examples.

See you next week,

Deepak

1. Product Interviews

2. Operating Well as a PM

I have previously written a long series (42 posts long) on Product-led Growth. You can get it in the email when you subscribe, or can find it on the newsletter home page.

10,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇

Subscribe now

Let’s dive in the topic now!

It was a Friday afternoon, and I was interviewing the candidates for a PM role. The first interview started with the usual question from my end — ‘Tell me about yourself’. The candidate went on to tell about everything he had done for the next 15 mins. At various points, I felt that he was going to stop now but he didn’t. I was an inexperienced interviewer at that point but even I knew this is a red flag.

I went to my manager and explained how the introduction was super long, and we spent most of the interview time (15 of 40 mins) on Introduction. We discussed few other things about the candidate but this was the highlight.

If you are wondering why having a long introduction isn’t right, here is the explanation. First, it shows lack of research and awareness on candidates’ part on how long the Introduction should be. More importantly, given that there would be actual interview questions post introduction, it shows that the candidates can’t estimate/anticipate timelines well, which is a crucial skills for PMs.

I am more interested in the 2nd part, and would spend rest of the interview looking for clues on whether that’s true or not. For example, I would double click on estimations aspect of product sense problems.

We decided to drop the candidate.

I never found preparing for PM interviews exciting! They always felt disconnected from the job, and what I wanted to really master is how to become an excellent PM. So the interview prep always felt like a necessary distraction.

That is until I stumbled upon an insight which I cover in this newsletter.

Because it has helped me so much, I am calling it the Master Mental Model for PM Interviews. Here it goes —

“Interview prep isn’t different from your actual job. Doing one can makes you better at the other, provided you understand the link between both. Further, you need to understand that there is a clear 'Why' behind the interview questions of every company. And this ‘Why’ is also unique to every company in some ways. Understanding that unique 'Why' is key to become better at the job and interview preparation.”

In the story I shared earlier, what I was more interested in was whether the PM can be trusted to come up with right estimations, and anticipate blockers — the real skills in a PM job. I wasn’t evaluating the PM on a set framework, my concerns were real.

And so understanding the unique ‘Why’ is the difference between a great and a good answer in an interview, and we will see later in this post around how to identify it and craft a great answer.

So where do we start? Let’s start at the real job. Let’s cover different stages of a product and what sort of real skills one needs to do a PM job well.

Stages of a Product, mapped to PM Skills

Every product goes through 3 stages - pre-PMF (PMF stands for product-market fit), growth, and maturity.

The pre-PMF stage is when you are trying to build the product 0-to-1. In this stage, you are looking at the market and deciding

• what product to build,

• what to launch in the first version, and

• what sort of iterations to do to get to PM/F

The growth stage is when you are looking to grow/scale the product that has gotten the PM/F.

The maturity is when the product is growing at the same rate as market is growing because it has hit the ceiling in terms of market share. This is the stage where you focus on diversifying your product portfolio so that you can find new avenues of growth.

Now that we have covered what the product needs in different stages, let's take a look at the skills needed in these stages

Skills in the pre-PMF stage

1. You need to be good with with market sizing so that you don't assess the opportunity wrong.

2. You need to be good with user research so that you can find and validate the customer problems in that market

3. You need to be good with problem-solving so that you can come up with few good solutions to the identified problem. You then further need to conduct research to validate the solution.

4. You need to be good with design, tech, and data so that you can get the right solution built at a faster pace than most people.

5. You need to be good with go-to-market strategy, which includes

   1. defining the first version of product to launch,

   2. pricing, and

   3. channels to launch the first version

To summarise, you need to be good at market sizing, user research, problem solving, understanding design + tech + data, and go-to-market strategy to do your job well in a pre-PMF stage.

Skills in the Growth stage

In the growth stage,

1. You need to come up with growth features that can lead to higher activation, and retention of the product.

2. You need to find avenues of revenue/monetisation in the product

3. You need to find new channels of product-led acquisition such as referrals, SEO/ASO, community, etc.

You still need to be good at shipping products which requires most of the skills in the pre-PMF stage — user research, problem solving, understanding design + tech + data, and go-to-market strategy.

Skills in the Maturity stage

In the maturity stage,

1. You need to deepen your product/market fit by constantly improving user experience at par or better than competition

2. You need to figure out other strategic avenues of product growth, like what product to launch next.

The skills required are more or less the same in pre-PMF + growth stage, however the mindset to operate is different because you don’t have much freedom to make mistakes as it can affect millions of user.

The above provides you a good sense of what’s needed to become a better PM in different stages of product. Let’s use this knowledge to prod the next question — is there a correlation between the real-job skills and PM interview questions?

Correlation between real skills and interview questions

PM interview questions (across levels) can be categorised into the following 8 buckets:

1. Opportunity sizing and guesstimates

2. Market entry

3. Product sense

4. Technical

5. Go-to-market strategy

6. Growth

7. Behavioural

8. Leadership

At an APM level, you are expected to clear these rounds

• Opportunity sizing and guesstimates

• Product sense

• Technical

• Behavioural

As you move up to a lead PM role, these additional rounds get added

• GTM strategy

• Growth

• Market entry

• Leadership

The Master Mental Model states that

“Interview prep isn’t different from your actual job. Doing one can makes you better at the other, provided you understand the link between both.

Let’s see the link between both by going through these various types of questions. You may have already done a rough linking between real skills and questions. By the time you are done with this post, you would be able to clearly see why working hard on your real skills can become a true differentiator in the interviews.

Let’s go via types of questions one-by-one and see the real skills mapping to those.

Market Sizing/ Guesstimates

Most people feel guesstimates are about looking at your mathematical abilities. But estimations are an important part of PM job. People who are good at quick estimations are usually good at market sizing, estimating efforts for a project, and estimating impact (and confidence) for a project, among other things. All of the 3 listed skills are pretty important to do a good job as a PM. And if you improve those 3 skills, your guesstimates will improve as well. You will be able to introduce nuances which a candidate following a rough framework can’t.

As an example, when a company asks you to estimate the market size of self-driving cars, they want to make sure that you won’t make a mistake in estimating market size of a new initiative on the job. That’s also why they care about the structure and approach that you take in these interviews, and not on the accuracy of the final answer.

Market Entry

While a growth stage company is usually focussed on acquiring more and more of a market, market entry is pretty relevant for startups and mature stage companies.

For example, a Google PMs need to spot which markets are attractive for Youtube to focus on. Should Youtube be building a custom experience for gamers or podcasters? A PM building a good case around a new market can open avenues of deeper engagement and monetisation for Google.

If you have done a real market entry problem at a job, you are able to touch nuances such as

• why is it important for the company now? Is there a sense of urgency?

• who are the key stakeholders and why their buy-in is important + how to get their buy-in?

• how much of difference does it make in overall P&L?

These are all practical questions you answer while doing a real market entry problem. Touching them in interviews goes on to show that the company can trust you with such problems in real-life.

Product Sense

Product sense is the most common round across companies. In a survey that I conducted earlier with this newsletter readers and discussed about in this post, product sense (also known as product design) gets asked in 100% of companies.

Common questions in the product sense round can be

• Designing a unique product like Design a blind-friendly refrigerator

• Favourite product like What’s your favourite product? How would you improve it?

• Improving a mature product. like How will you improve Google Maps?

Product sense questions are common because these questions directly map to the day-to-day skills of a PM across companies and product stages. Let’s go through one of these questions like Design a blind-friendly refrigerator

A popular recommended framework to solve such problems is CIRCLES

And we usually try to go through every step of this framework in the interview to come up with a solution.

But you can also organically arrive at a similar framework if you ask yourself on how to design good products.

Imagine that you have estimated the market size correctly, and have concluded that your company should design the blind-friendly refrigerator.

• The immediate next thing to do in real-life PM is to conduct user research to understand users well. The goal of the research is to identify the needs and preferences of a target user persona. This is tested in the product sense round of the interview. You are expected to talk about the customer (blind people), and their problems/needs when it comes to refrigeration. This ties to the Identify and Report step in the CIRCLES method.

• Once you have concluded the user research, you end up prioritising the top user problems you want to solve first in your job. You are expected to do the same in the interviews (Cut through Prioritisation - CIRCLES

• After prioritisation, what do you do? You sit with the team and create solutions, get designs done for the solution, do the tech estimation, and re-prioritise features to finalise the sequence of development. You are expected to do the same in the product sense interviews. The same sequence of steps are followed in a product sense interviews — List solutions of the problem you have prioritised, Evaluate trade-offs (effort-impact framework). At this stage, you can also create mock-ups, do a quick effort estimation, and then reprioritise what to build. The interviewer gets to know how well you can do these steps because they reflect how well you would be able to do it in the real job.

Once launched, PMs need to assess whether the launch was a success or failure. In the product sense round, PMs should define success/failure criteria. There are additional steps that you should include to impress the interviewer in a product sense interview.

PMs who are good at their job also end up doing richer discussion in these interviews, provided they don’t hurt themselves by sticking too much to a framework and are able to show their experience while answering a question.

The other two common questions in product sense are also relevant in real jobs. Once the product has entered the mature stage, the problems that often arise are like 'how do we improve this particular product'. Similarly, the taste of a good PM is their judgement around good vs bad products. That's where the questions around favourite, and least favourite product comes in the product sense rounds.

Technical Round

PMs work quite closely with developers, and a significant chunk of Product development is engineering. And so the technical understanding of a PM is gauged by allotting a different round for it because it is quite important for PMs to understand the choices made during development of a product.

We should note that PMs don’t need to code to do their jobs well, and so coding interviews don’t happen are usually involved in technical round. PMs need to understand the technical system design well. They also need to be comfortable with doing root-cause analysis of a product experience problem, happening due to a technical issue. And finally they should be able to communicate everything effectively across the board.

All the three skills — system design, RCA, and ‘explain it in layman language’ — are tested in the technical round of the PM interview. If you are looking for a useful resource to build your technical acumen, I wrote a book to solve this same problem for PMs without technical background — https://www.amazon.in/Tech-Simplified-Entrepreneurs-Deepak-Singh/dp/9355664990

Go-to-market (GTM) Strategy

One the product is built, it needs a GTM strategy to reach the right set of users. This usually isn't expected of APM candidates. But as you gain some experience, this is expected of you.

GTM usually involves the same steps as the real-life PM:

• taking a decision on how the launch should happen - Experiments vs Beta vs Full rollout

• finding right channels for acquiring initial customers. This again requires an in-depth understanding of growth, required in senior roles.

• defining the right goal for initial launch.

• pricing the product

The GTM interview questions can be around the launch plan including getting to first N customers, or pricing, or a combination of both.

Growth

Growth round can have questions around tackling a growth problem like low activation low, retention, or engagement. The round tests for quality of solutions you can come up with when faced with such a problem through product.

All of the above problems are very similar to the real-growth challenges faced by all products at one point or another.

Behavioural and Leadership

This should be self-explanatory as most of the questions in these rounds are around things you have done in previous jobs. The better you have behaved/worked in your previous jobs, the better will be your answers in this round.

The above topics should give you a fair idea on how PM interviews map to real PM skills.

Let’s move to the next part of the mental model

“Interview prep isn’t different from your actual job. Doing one can makes you better at the other, provided you understand the link between both. Further, you need to understand that there is a clear 'Why' behind the interview questions of every company. And this ‘Why’ is also unique to every company in some ways. Understanding that unique 'Why' is key to become better at the job and interview preparation.”

The ‘Why’ Behind Product Interviews

Most of the Interview Prep literature starts and end at - "Here are the problems that gets asked in Company A, and Here is how to prepare for it". And there is a good reason for it. What's the point of asking 'Why' certain questions get asked in the Interviews of a particular company? It might distract you from preparation. So isn’t it pragmatic to just prepare and crack the interview?

No!

You can become much better at Interviews over time as you understand that 'Why' better. PM interviews are subjective, and the ‘why’ provides you an insight into where to focus in the limited time you get into an interview.

So how do we understand the ‘Why’. There are two factors that affect ‘Why’ — role, company. Let’s start with role/level.

Role Relevance

The same question can be asked at various levels in an interview.

At an APM level, companies expect candidates to display the general skills needed for a beginner role such as APM. But as you move up the ladder, PM Interviews can change depending on the scope you manage in these roles. We have already seen how some rounds don’t happen for APM/PM level, but start happening in senior roles.

But even these senior roles have varied scope. For example, a GPM can have a very different scope than Director of Product. That is where PM interviews become confusion. How are we supposed to know what skills to display for the level we are interviewing?

It's almost impossible to provide a strict guideline across levels, but it is possible to predict by asking few question to the HR or hiring manager in the pre-screening rounds. These questions should try to understand why the company is hiring a person in such-and-such role, and what sort of scope do they provide with this role. In addition to these, understanding the culture of the company helps in crafting the answers.

For example, suppose that the company has been facing user growth problem due to low retention and wants to hire a product lead for this role. So while answering a simple question like ‘Which is your favourite product, and why?’, you can touch upon stickiness or retention aspects on this product while telling about it.

Company Relevance: Stage and Culture

Let’s try to understand how company’s stage and culture affect the ‘Why’ by taking an example of Booking.com. We know that it's a mature stage, consumer company in travel.

Booking needs PMs who are

• technical because it’s a complex, mature product

• good with experiments since the untested features can’t go live

• good with process, collaboration and stakeholder management because the large company size demands these skills to get projects moving

• good with product sense/ ideation so that they can come up with features to improve a mature product such as book

You can see these reflecting in the questions that Booking asks (Source: Glass-door)

1. Process oriented questions like

   1. What is your process for working with Engineers?

   2. Can you tell something about the impact you made in your previous role, and why do you want to leave your current company?

2. Stakeholder management questions like

   1. Tell about a situation where you had to manage upwards on a product feature where you had a disagreement.

3. Product sense questions like

   1. Add a feature to your favourite product

4. Data questions like

   1. How would you explain the changes in XX (certain metrics)?

5. A/B testing questions like

   1. How would you test a new feature? How would you decide on the best approach to decide on the traffic that should be tested?

6. Technical questions like

   1. Tell me about a time when you had technical influence over a product

Compare it to a B2B product.

• B2B products usually require an understanding of complex workflows. You have to think about how a simple change can affect the customers who are paying you. So it becomes essential for such PMs to display this acumen. In addition to that, you also need to understand the customer profile (SMB vs Enterprise), and GTM strategy.

• B2B product management also focuses a lot on stakeholder management because your customer is an active stakeholder in the process. So you often get questions like

  "Tell me about your relationship with Engineering: what do you like most about working with Engineers? How do you handle disagreements/conflicts between yourself and engineering? Using an example if possible. " - Salesforce

• B2B product management relies on set process like agile and scrum because they oftentimes need predictability on their roadmap. So you can get questions like these in the interviews.

The above should provide you a good sense of the unique ‘Why’ behind certain questions asked by certain companies. If you can understand the role well and company’s stage and culture, it becomes much easier to provide relevant answer. It also becomes easier to anticipate questions. For example, someone hiring for a platform PM role would be much interested in seeing if you have a strong technical aptitude and would ask questions around that.

If you have suggestions, let me know how I can improve the future posts - https://forms.gle/q6b67rqVwwz5saaq7

What's Next in the Product Interviews Series

I plan to write detailed interview guide for all types of product interviews. Each post will have the following structure

1. Relevance of this type of questions

2. Companies that love this question

3. Most popular questions around this

4. Approach or framework to solve this question

5. One End-to-end example of each sub-type

6. Key Mistakes and examples of such mistakes

7. Test your knowledge of this topic (beginner vs advanced score)

8. Subjective Questions Database for further practice

9. Books around the topic (I have read vs haven’t read and rating)

10. Articles and videos around the topic and my rating

All of these posts will be free for the foreseeable future, so subscribe to stay updated and never miss a post!

Subscribe now

Thank you for reading :)

If you found it interesting, you will also love my

1. Advanced Growth Program for PMs

2. Product Sense and Strategy Program for PMs

3. Bestseller Book ‘Tech Simplified for PMs and Entrepreneurs’

For the new ones here, do check out the popular posts that I have written recently if you haven’t

1. Assessing the effectiveness of PMs

2. Product metrics: to measure or not to measure

3. Necessary Skills to Enter PM Roles/ Crack PM Interviews in India

4. Hidden Layers in Product Management

5. How to Crack the Technical Round of PM Interviews

10,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇

Subscribe now

Let’s dive in the topic now!

In last week's post, we covered Mistakes to Avoid in Entry Level PM Interviews. In this post, we will focus on the mistakes in Lead PM roles, i.e., SPM, GPM, etc. Please note that these mistakes are very different than the ones people make in entry-level APM/PM roles.

Lead PM roles are often PM manager roles where you manage a PM team. The leadership skills and scope of the role change significantly between PMs and lead PMs. A Product Manager follows the product strategy and implements the product plan within the context of that strategy. The Lead Product Manager is responsible for the product strategy. So, the expectations in the interviews are quite different. Candidates get down-leveled to PM roles after interviews even if they have done lead PM roles in the past because they don't understand what they have to index upon in these interviews. They try to apply the same preparation material, thinking that they just need to work extra hard this time.

So, what are these mistakes, and how can we avoid them? The critical mistakes in senior product interviews are the ones that signal to the interviewer that you can't manage this role. The first one, and the most important one, is providing a bookish answer.

1. Providing a Bookish Answer

Using frameworks is great for structuring and touching upon everything in the limited time you get in the interviews. But when you latch on to the frameworks to find the solution to a product problem, it might result in a bookish answer. A booking answer is the one that doesn't account for how real-world product development happens.

To explain this better, let's take an example. CIRCLES, coined by Lewis Lin, is one of the most overused frameworks in product design/ product sense rounds.

When you use CIRCLES in a real interview setting, here is what a bookish interaction might look like. I have intentionally kept it shorter so as not to make it too boring and to convey the central points.:

Interviewer: You are a PM at Google. Design a product that grandparents can use to connect with grandkids remotely.

Interviewee  [Clarify and Identify the Customer]: Sure, we have to design a videoconferencing app that both grandparents and grandkids use to connect with each other. When we say kids, I assume that we are talking about those aged 6-12. Children below six aren't allowed to use devices on their own. Can I make that assumption?

Interviewer: Sure!

Interviewee: I would take some time to jot down my thoughts..

<90 seconds later>

Interviewee [Report Customer Needs]: So for kids, we know that there are some pain points they face while teleconferencing with their grandparents:

1. They are usually distracted and get bored easily while chatting

2. They move around, so the sound and video reception may get poor due to the movement of grandparents

Grandparents also have a set of pain points

1. It's hard for them to use apps because they have a steep learning curve. So, the app design needs to be simple

2. Some of the older grandparents are visually challenged and have auditory problems as well.

Does that sound right?

Interviewer: Sure, please move ahead!

Interviewee: [Cut through Prioritisation] I will prioritize the pain points now based on severity and % of users affected by it.

1. Distracted while chatting: This is a key problem since conversations will only be high quality if kids are focused. So, it's important to solve this pain point. I would put this under a high-severity bucket.

2. Sound and video reception while kids move around: Important to solve, but medium severity.

3. Simple app design: This affects all older people, so high severity

4. Visual and auditory challenges: These are important problems, but they don't affect everyone, so I classify them as medium severity.

I will now create the solutions for these pain points.

• getting distracted

• app design needs to be simple

Interviewer: Sure!

Interviewee: [List solutions and Evaluate Trade-Offs] We need to create an app

• to keep kids engaged

  • introduce bright colors, filters to try out, etc., to keep the kids engaged [low effort]

  • introduce games that they can play together while video chatting because often, kids love games [high effort]

• design the app with large fonts and big buttons so that it is easy to use for grandparents [low effort]

Here is what the screens would look like.

<adds screens>

Interviewee: Now, I will define success metrics. To start with, we can focus on acquisition and customer satisfaction scores (ratings at the end of the call).

And that's a wrap!

Usually, an answer like this is okay at an APM or PM-level interview. The key reason is that when you are new to product management, you need to master structure and learn everything needed to answer these questions, such as metrics, how to create mock screens using elementary design principles, prioritization, thinking from a user perspective, etc. Displaying all of these in a limited time is good enough to get into the roles.

However, as you move up to a lead PM role, merely using the frameworks limits your product thinking.

So, how should one answer questions at a Senior PM level? Solve them the way you would solve them in a job. You should still use the framework to maintain structure and touch upon everything in the limited interview time.

Specifically, here are things one should include:

1. Who are we, and why does it matter to us? — Google has a portfolio of video conferencing apps already. Covid has accelerated the adoption of such apps, so they have become critical to users' lives. Usually, the richer the connection, the better the experience is in this case. So far, the industry has built basic features and solutions for latency, video, and audio quality in low bandwidth, chat, etc. From here on, innovation lies in solving for specific segments. Kids and grandparents are an underserved segment, and if Google solves it, it can create the right impact on users' lives and also give Google a lead in this segment.

2. Choosing a goal — If we know the why, it will tie to the goal. The goal here is important to clarify so that you can work backward to the pain points, solutions, prioritization, success metrics, etc.

3. User segments — In beginner-level PM interviews, segmentation on the basis of age, gender, and geography is okay-ish. But when you build products in real-life, you try to think of behavioral segments (personas with specific attitudes and behaviors) and solve for them. So, even in the interviews, segmentation needs to happen based on behavioral traits.

   For the particular interview question, we can segment kids based on their behavior. Kids between the ages 4-8 (Segment1) may prefer playing games, listening to stories, watching movies, etc. together, whereas kids between 9-12 (Segment2) like to tell what happened at their school and play games that are complex and may not be suitable for grandparents. So these two can be important segments.

   The needs for Segment 2 can be solved by Zoom or Meet. Therefore, kids between 4 and 8 are more suitable for a richer connection experience, and we would select them as our segment.

   Similarly, not all older people are digitally challenged, and those who are won't be able to do digital stuff like playing games with kids wouldn't be able to engage in a rich manner, which is a large enough segment based on my interaction with older folks, so we can focus on the segment that has digital awareness and leave the rest.

4. Pain points and solutions— You can go on to list pain points for both S2 and digitally aware grandparents.

   For one, language skills can vary between generations and locations. Grandparents may be more fluent in Hindi/ Spanish, while kids may be more fluent in English. So, there is a language barrier to be solved. This also helps us determine that going first after visual games and visual stories might be more useful.

   Beyond chat, they also need other ways to interact so that kids aren't distracted and bored. Entertainment in the form of stories, games, and videos works fine.

   Beyond entertainment, grandparents also want to teach some useful skills like language, maths, dance, solving puzzles, etc. We can build rich experiences for teaching basic reasoning, maths, creative pursuits, etc.

5. Prioritization — While CIRCLES advocates prioritizing pain points before the solutions, in real-life product management, prioritization doesn't come before a broad solution exercise. If you want to use the effort-impact method of prioritization, add possible solutions to pain points and see which are the most appealing ones.

6. Metrics — Since we are talking about the quality of experience (goal), metrics related to experience, such as ratings, engagement, feature adoption, etc., should be important. Mention the sanity metrics for experience as well, such as crashes, screen freeze, latency, etc.

To reiterate, solve the problem like you would solve in real life. That would yield a much richer answer.

2. Conflating Corp and Product Strategy

Corporate strategy is different than product strategy questions. Corporate strategy questions focus on choices that improve the competitive positioning of the corporation as a whole. In contrast, product strategy focuses on choices you must make in a particular product line to meet BU objectives.

A few examples of Corp Strategy questions are

• Should Google acquire Garmin?

• Should Amazon enter Indonesia?

• What should Microsoft build next?

• Should Flipkart get into Food Delivery?

In corporate strategy, you need to focus on the company vision, what the company is good at (like MSFT is good at building platforms), and the competitive landscape.

Product strategy questions are more around a singular product.

• How would you launch a teleportation device Google just built?

• How should Google Cloud catch up with Azure and AWS's market share?

• How would you increase Slack's revenue by 10x in the next year? Notice here that Slack is a singular product right now and not a portfolio of products.

In product strategy, you also need to focus on the company vision, what the company is good at (like MSFT is good at building platforms), and the competitive landscape. In addition to these, you also need to do a launch plan, roadmap, and success criteria since they are crucial to your product's success.

3. Showing Stress

While everyone gets stressed once in a while, you need to be able to stay calm in difficult situations as a lead PM. Your team's morale depends on how calmly and effectively you handle a particular situation. That is why a few companies throw quite difficult problems to a candidate to see if they can maintain their nerves.

So, how do you make sure that you are staying calm? By doing mocks. You need to observe yourself in mock interviews and watch out for when you start speaking fast and incoherently. Most people's speech and breathing pattern changes when they get stressed.

Like most things in a PM job, knowing this isn't enough. You have to practice to be better at spotting and managing stress.

Some effective strategies, backed by science, to get back on track are:

1. Take 4-5 deep breaths

2. Take a sip of cold water

3. Revisit what you have done so far to look for clues on how to get back on track

4. Not knowing the products you have built in the past deeply

At lead PM level, you are supposed to know the whole story of the products you have built. The whole story includes —

1. Historical context around why the company built it and when it started

2. Different versions, when they launched, and how they performed

3. Technical system design of the product

4. What you shipped and how it performed

Cover all of these for the products you have built over the last 2-3 years, and you should be good. Write it down somewhere for future reference, both for interviews and learnings, as well.

5. Not Being Assertive in The Interview

While it's important to be likable, you need to be the person you would be in your real job. Show your personality and its unique traits, including the fact that you can make opinionated decisions and hold your ground when the interviewer pushes back.

Jobs at the lead level are a lot about making a convincing argument for something you believe in. So, while the interviewer is gauging you on the technical skills of a PM, they are also looking for clues about whether you can take initiative and convince others to follow.

If you improve upon these five pointers listed above, the chances of getting a lead PM role significantly enhance.

That would be all for this post. Do let me know how you found it, and see you next week where we cover the mistakes made in executive-level interviews.

🔥 Launch Alert 🔥

"I learned very early the difference between knowing the name of something and knowing something." — Richard Feynmann.

When I started pmcurve.com, a pressing challenge for me was a lack of awareness among product managers on how well they knew a certain topic. Even the best PMs doubted themselves, and many had flawed confidence in their abilities as well.

The reason for the above is the lack of objective assessments and benchmarking. It also makes it hard for PMs to actually know that they have learned something while going through a course/cohort.

So, I wanted to solve this problem. 

Growth Playground for PMs has 600+ application-oriented, objective 

assessments. PMs can now learn everything about product growth and objectively evaluate themselves on how well they know it in the Playground.

Further, practicing such problems helps in remembering and applying the concepts better.

Launching on 6th Jan 2024, Playground is a powerful way for PMs to master growth. Beyond objective assessments, other aspects make it the best way to become a top-rated growth PM.

The anticipation is building, and 250+ people are planning to join the launch event! I hope to see you there 😁

Join the Launch Event

For the new ones here, do check out the popular posts that I have written recently if you haven’t

1. Assessing the effectiveness of PMs

2. Product metrics: to measure or not to measure

3. Necessary Skills to Enter PM Roles/ Crack PM Interviews in India

4. Hidden Layers in Product Management

5. How to Crack the Technical Round of PM Interviews

10,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇

Subscribe now

Let’s dive in the topic now!

Good morning,

I am writing a 3-part series on mistakes to avoid in product interviews.

• Part 1 is about the entry level roles(APM/PM)

• Part 2 is about the mid-level roles (SPM/GPM)

• Part 3 is about executive roles (Director and above)

Part 2 and 3 would be future posts that would be published within the next couple of weeks. Let’s start with the mistakes that aspirants make in entry level roles in this post. But before that, I have an announcement to make!

🔥 LAUNCH ALERT 🔥

The Growth Playground for PMs is a learning product I would be launching on 6th Jan’ 2024. Playground is a unique and effective way to master growth for PMs. It's rigorous, objective, and closest to real-life product management.

Couple of days back, I created an event on LinkedIn that you can join and the response has been really good. So looking forward to it :)

RSVP the LinkedIn Event

I hope to see you there 😁

Let’s move on with the topic at hand — mistakes to avoid in entry level product interviews. I have seen PM aspirants making these mistakes again and again, so I thought it may be useful to list it down somewhere.

Mistake #1: Not doing mock interviews

A lot of candidates never feel prepared enough to do mock interviews. They go through all the resources, such as paid courses and forums, and do some practice problems themselves. But they always shy away from mock interviews.

Many candidates who feel ready still don't do mock interviews because they don't find someone senior available to take these mocks.

Mock interviews are essential, and it's important to understand where a mock interview helps you in the preparation of product roles. It helps in three ways:

1. It time-boxes the problem-solving and helps you figure out where you spend most of your time. You don't need an expert to figure this out. You can record and replay the interviews to figure it out yourself.

2. It helps you find problems in communication. Many times, we know things but can't communicate them effectively. Other times, we have an odd habit of repeating a word or phrase that can annoy the interviewer.

3. It helps you get a third-person perspective. This is where an expert interviewer can be more helpful in finding what you don't know. However, you can still practice and be perfect in what you already know. If you become perfect in what you already know, the expert feedback can be pretty sharp in your case, and you won't take much time.

Based on the above, you can sequence mock interviews as practice yourself -> do mock with a peer -> do mock with an expert. 

An important thing to understand is who qualifies as an expert. An expert has taken PM interviews for their companies in the past. These PMs know how to evaluate a candidate using a rubric (more on this in a future post). Therefore, a junior PM doesn't qualify as an expert since they don't get to do such interviews. 

While there isn't a rule of thumb on how many mocks you should do, aim for 8-10 peer mock interviews and 2-3 expert interviews to grasp various types of questions and frameworks.

Mistake #2: Taking Youtube Interviews as the benchmark or source of truth

YouTube, in recent years, has exploded with sample product interviews. So it is natural for aspiring PMs to go there and try to imitate such interviews. While they are helpful in many ways, these YouTube interviews can also do more harm than help, depending on how you use them:

1. The interviews posted on YouTube are often scripted. In a scripted interview, the Interviewee knows the question in advance and prepares for it. Such interviews may come as a wow! An aspiring PM seeing these interviews may start feeling under-confident because they feel they can't think or perform at the level the people appearing in such interviews are. The under-confidence can lead to poor performance in product interviews.

2. Sometimes, the reverse can happen. The interviewees, especially from more prominent companies such as FAANG, can come directly to such sessions and may be out of touch when it comes to interview preparation. They have prepared for an interview years back but haven't practiced because they don't need to. Trying to decode such interviews can be frustrating because they don't adhere to frameworks. 

3. Strong PMs rely greatly on their experience and domain knowledge to solve interview problems. They feel comfortable breaking some of the popular frameworks and focusing on what matters most for product decisions, even in an interview setting. Such interviews aren't useful at the beginner level. 

So, how should you watch YouTube interviews? Try to observe interviews done by candidates who recently cleared APM/PM roles. Look at how they apply certain frameworks. But at the same time, don't penalize yourself if you didn't think of some of the things the Interviewee did.

Mistake #3: Asking too many questions

PM interviews are quite similar to consulting case interviews. Both offer open-ended questions that you are supposed to solve using a structured approach and frameworks. However, there is a big difference between case interviews and product interviews, which I wrote about earlier in the post 

There is a big difference between the case interviews and product interviews, which many people don’t even realize during the interviews. The distinction becomes especially important in the product design round.

• In ‘case interviews’, you are supposed to ask the right data points to solve the problem.

• In product interviews, you are supposed to make intelligent assumptions and narrow them down to the solution. In short, you can’t ask many questions in product interviews.

This doesn't imply that you are forbidden to ask good questions in product interviews. Just don't try to get direct answers from the interviewer. Do not ask directly,

• Who is the user?

• What is our main goal?

Rather, 

• Ask Whether your assumptions are correct around product goal, market, user, etc.

• Ask clarifying questions if the questions are too broad, like 'designing a modern fridge.'

Unlike consultants, PMs don't have clients who can give them direct answers. PMs need to make intelligent assumptions to do their job and validate these assumptions. That is what your interviewer expects from you. 

Mistake #4: Not understanding the product well

Not understanding the product you are interviewing for can mean many things. The worst among those is not knowing what the product does. Candidates failing to check the app and website before getting into the interviews are definitely setting themselves up for failure. So, do check the product before getting into the interview.

If you want to get an edge, you need to go beyond a basic understanding of the product. You should aim to understand the target users, why someone should prefer this product over alternatives, market size, competitors, etc. It allows you to speak more comfortably around questions such as 'What do you think of our product and how can we improve it?'. It also allows you to ask intelligent questions such as, "Given that the existing players are struggling with growth, what's our strategy to grow the product?"

Mistake #5: Not preparing for tech knowledge and product development processes

At the entry level, you aren't supposed to know much about tech and product development processes because you don't have direct experience. That said, I would be lying if I said that the interview panel doesn't prefer the candidates who exhibit knowledge in these areas.

As an APM/PM, your primary job is to work with designers and engineers to ship the product. Get familiar with technical concepts and how product development processes such as Agile, Sprints, Waterfall, etc. work. It can give you an added advantage.

Since product management is a lucrative domain to get into, the competition is getting stiffer over time. So build up all the advantages you can. I will share resources to prepare know-how of the product development process in a future post, so stay tuned. 

On technical literacy - if you come from a Computer Science (CS) background and have paid attention to UG courses, you may already have decent knowledge for these interviews. If you don't come from a CS background, don't worry because I wrote a book for you 😊 — Tech Simplified for PMs and Entrepreneurs.

The book is written in simple language and can be quickly done within a week if you spend 3-5 hours daily.

Mistake #6: Lack of structure in non-product questions

Most candidates who prepare well for product questions stay structured in those questions. However, as soon as they get a non-product question, they feel as if they need to give a natural, free-flowing response.

While a natural, free-flowing response is the best kind, it shouldn't come at the expense of clarity and brevity. To be safe, introduce a structure in even these responses. 

Some of the useful frameworks around this are STAR and the Minto Pyramid Principle.

The STAR method gives you a format you can use to tell a story by laying out.

• Situation: Talk about the situation you were in, like playing cricket for a university where I was the Captain.

• Task: Describe what the task was in that situation, like "Teammates weren't coming to practice sessions regularly."

• Action: Explain exactly what steps you took to address it, like "I did 1:1 conversations with these players to understand what prevented them from coming to sessions regularly."

• Result: Share what outcomes your actions achieved, like "We figured that players didn't have predictability in schedules so we created one that was consistent and predictable. As a result, the participation in these sessions improved significantly."

You can read more about STAR and Minto Pyramid Principle online.

Mistake #7: Placing a premium on intros

I have interviewed multiple candidates who spent the first 10 minutes of the interview answering the question — "Tell me about yourself." Spending too much time on these questions shows a general lack of research and awareness on the candidates' part.

Candidates often focus too much on crafting the perfect introduction or 'tell me about yourself' response. While intros are essential, they are just a tiny part of the interview and should be treated as such. As a general rule of thumb, spend some time crafting an intro of 60-90 seconds telling your story.

Part 2 Preview: Avoiding Mistakes in Senior Product Manager (SPM/GPM) Roles

In the next part of this series, we'll dive into the common pitfalls faced by candidates interviewing for Senior and Group Product Manager roles. These roles come with their unique expectations and challenges, and understanding these can be key to your success in these interviews.

Stay tuned and have a good day :)

Do hit a like if you loved the post!

For the new ones here, do check out the popular posts that I have written recently if you haven’t

1. Assessing the effectiveness of PMs

2. Product metrics: to measure or not to measure

3. Necessary Skills to Enter PM Roles/ Crack PM Interviews in India

4. Hidden Layers in Product Management

5. How to Crack the Technical Round of PM Interviews

10,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇

Subscribe now

Let’s dive in the topic now!

In the last post, we covered the knowledge tree for PMs. Here is an illustration of the same for recap,

At the end of the post, I mentioned that various books can be mapped to these skills. A right mapping would prove useful for PMs trying to get better on a particular skill.

While there are plethora of books on each topic, I am going to recommend those that I have read myself. These books have helped me immensely throughout my career as a PM.

Further, a laundry list can be quite overwhelming so we will limit the recommendation per topic to top 1-3 books. At the end of the post, there is a master diagram provided which contains all the recommended books.

Recommended Books to Learn Skills

Let’s cover the skills one by one

1. Written Communication — A book I like most around written communication is On Writing by Stephen King. Stephen king is a terrific writer. Half of the book is writing guide, the other half is memoir.

   Another foundational book around writing is The Elements of Style by Strunk and White. It doesn’t matter which one of these two you pick first, but you need to finish both to have a sense of the art and science of writing.

2. Verbal communication — The key problem in verbal communication is lack of structure. And a book that helped me solve this problem is The Pyramind Principle by Barabara Minto.

   Another book around having difficult conversations is The Crucial Conversations. Crucial conversations helps you with the right mindset to be in during stressful situations. While it’s difficult to implement everything the author talks about in the book, it definitely helps you shift your mindset to the extent that you would avoid blunders in stressful situations.

3. Negotiation — The only book I am going to recommend around negotiation is Never Split the Difference. The book is written by a former FBI negotiator and is a wonderful read.

4. Org Psychology — I haven’t come across anything excellent in this area, and so don’t have a strong recommendation. The book that helped me decipher various org cultures and why each culture can succeed despite the differences is No Rules Rules by Reed Hastings, the founder and CEO of Netflix.

   Another good book around this is How Google Works by Eric Schmidt, the former CEO of Google.

   But while these books provide you an insight into conscious choices made by founders/executives to create a strong culture, it doesn’t talks about the mechanics of org psychology. There is much you have derive yourself.

   There are many books which talks about why some organisations fail while others succeed. One popular one in this category is Good to Great by Jim Collins. But while books such as Good to Great are an entertaining read, their research methodology relies on narrative of leaders and is unreliable because of narrative fallacy. So read them because there is always something to learn, but don’t take them as truth. Reading a book about a single company gives a better view since you can see whether the whole story makes sense, and also vet what’s written to some extent by talking to people there.

5. Behavioural Psychology — The definitive book you need to read on behavioural psychology is Thinking Fast and Slow by Danial Kanhemann. It’s a dense read and took me longest to finish among all I have read so far. However, the effort is worth it since it gets you started on behavioural psychology. Another one of my favourites is Influence by Robert Cialdini, which talks about the psychology of persuasion. The third book on this list would be Hooked by Nir Eyal, which talks about habit formation in products.

6. User research — User research is a key area for PMs and design teams. The recommended books on this topic are Don’t Make Me Think, The Mom Test, and Continuous Discovery Habits. The three books are quite different from each other.

   ‘Continuous Discovery Habits’ talks about setting up a system to conduct research, whereas ‘The Mom Test’ is focussed on how to conduct user interviews. ‘Don’t Make Me Think’ is an entertaining read which talks about the importance of usability tests and how to conduct them well. What I like about these 3 as compared to other books is that they focus a lot of ‘why X is important’/ They don’t prescribe something, but rather take a first principled approach — crucial in a truth-seeking endeavour such as user research.

7. Data analysis — Data analysis is where I have read a whole lot of books, primarily because I have built and led product analytics and data science teams in startups. And unfortunately, I haven’t found many excellent books in this domain. How to Lie with Statistics is a short and entertaining read which also helps you spot common problems with data analysis. A decent book around metrics of different companies is Lean Analytics, but it falls short on how to think holistically about data. These recommendations aren’t enough to master product analytics imo, so you should seek out more resources on the internet after you finish these. Here is a list of such resources that I covered in an earlier post.

8. Industry analysis — Industry analysis is best covered in two books: Competitive Strategy by Michael Porter, and 7 Powers by Hamilton Helmer. While the first one talks about how to study the structure of the industry and its 5 forces, the second one focusses more on how companies build moats/defensibility in their models. ‘Competitive Strategy’ by Porter is a heavy read, so if you are short on time, you can pick Understanding Michael Porter by Joan Magretta which is easy to understand.

9. Project management — Of the numerous books published over Scrum and Agile, I find the one by Jeff Sutherland apt for PMs. He is the co-creator of Scrum. The book is titled Scrum: The art of doing twice the work in half the time.

   Another excellent book around project management is The Phoenix Project, which talks about development and software shipping challenges. The whole book is a elaborate story of an IT company, so it’s an enjoyable read.

   Possibly, my favourite book on project management is The Art of Project Management. The book was written in 2005, and some of the portions may be outdated. However, it does a fabulous job of bringing all elements of project management together — teams, people, design, development, testing, leadership, politics, etc.

10. Design — Design of Everyday Things by Don Norman is a clear winner when it comes to UX design. I am yet to find a good book on UI design, so if you have a recommendation, please share.

11. Tech Understanding — PMs don’t need to code, but they definitely need to understand tech terminologies and high-level system design. Tech Simplified (I wrote this) would be my recommendation as there aren’t many books on Tech for PMs. Swipe to Unlock is another good read, however it talks more about tech business and strategy (and a little bit of how tech ties into that).

12. Creativity — I would leave this section blank as while I have read books on this topic, but they haven’t moved me from place A to B in terms of creative skills. I am currently researching this one skill for my next project, so will keep you updated.

    A good, informative read around what makes organisations creative is Creativity Inc which writes about the creative methods and culture at Pixar.

13. Business 101 — There are far too many books around business, so it’s hard to pick top 3 recommendations here. Each business book brings a new perspective, yet limited in some way. The limitations also arise because business landscape is continuously evolving.

    For those who haven’t done MBA, a good place to start is Business Model Generation.

    Playing to Win should be definitely read by anyone building strategy to launch/optimise product lines. The Innovators Dilemma is another great read for people working in tech. It is thought-provoking, and quite original in thesis.

Here is a good visual diagram to help you save and remember

What books are missing in the above?

If you look at the diagram, you may realise some of the most popular books in PM missing there:

1. Cracking the PM Interview by Jackie Bavaro — Even if I have read the book and love it, it is useful for cracking the interviews and not for real-life product management. So it doesn’t make the list.

2. Inspired by Marty Cagan — If you are an aspiring or beginner PM, you should give it a read since it makes you familiar with product terminologies and common norms of product management.

   I would be in the minority of PMs who started the book but didn’t finish it. I put this book in the same category as Good to Great, which can be an entertaining and useful read but can be a sweeping generalisation. Since the excellent product teams have diverse culture, these generalisations don’t apply at all places.

Few of my favourite books like Crossing the Chasm didn’t make the list because I had to pick books that cover enough breadth and depth for PMs to get comfortable in those areas. Once that is done, you can go ahead and read more as you like.

What areas are missing in the above?

One key area that’s missing in the above is product marketing which focusses on GTM and pricing. That is because I found it hard to map it to a particular competency. It would be closest to business/strategy, but I would like to keep it as a separate competency. If you are interested, here are top 3 recommendations:

1. Hacking Growth — covers basics of growth

2. Obviously Awesome — covers the key foundation of GTM, aka positioning.

3. Innovating Monetisation — covers pricing

How should you read these?

Reading books is difficult for many people. Reading so many is even more difficult. Start small and read what interests you.

Lately, I only pick books that tie to a problem I am solving in my life. That way the motivation is quite high to read. So if you are already a PM, map books to a problem you are solving or your areas of improvement, and you should have enough motivation to read.

Another alternative to reading the books is listening to audiobooks, or watch any lecture delivered on the topics by the author. Both are good to get a grasp on key concepts quickly.

Going forward, we are going to cover basics of all the skills listed above. So subscribe and keep reading the Growth Catalyst :)

Do mention the books you have loved in the comments, skills they map to and why so! It will help me create a richer list by going through them.

Happy reading!

Deepak

Before we move forward, I wanted to make a quick announcement. Today is the last date to apply for pmcurve Product Growth course, relevant for existing PMs and Founders. This would be the last live cohort that I am planning to do around product growth this year. So if you were considering this, now would be the time to act. Here is the course page from where you can apply. Here are the raving reviews of the course from 1st cohort.

Product Growth Course

Moving forward,

Here is the recorded session from today around product sense.

You can find the slides used in the session here.

Here are the key takeaways from the session:

1. Product sense is the ability to make right decisions about which changes would bring most value to users. It is usually a two-step process: 

   • Identify problems and their significance

   • Design the right solution for them

2. If I had to pick an Amazon leadership principle to explain product sense, it would be ‘leaders are right a lot’. This is something which is counterintuitive to majority of PMs because they believe customer obsession = product sense.

3. Product sense is hard to master because rules, formulas and principles simply will not work. The forces that determine the outcome are constantly shifting when it comes to building products. Things keep changing everyday in the product world, and what worked yesterday won’t work today.

4. Product sense is also known as product thinking, product judgement, product design. It is essential for PMs to learn because 100% of product interviews test product sense.

   

5. Product sense can be applied to everything in life. I applied it to write a book and design a live course. Check the video and slides for details.

6. When we deconstruct what’s needed to build strong product sense, we come up with 6 skills

   1. To identify problems and their significance, we need (step 1)

      1.  Understanding users

      2. Understanding current industry and business models

      3. Frameworks to identify/frame problems

   2. To come up with solutions (step 2), we need

      1. Understanding the tools to solve the problem (Growth, Tech, Design)

      2. Range of knowledge to find analogies from similar problems others have solved

      3. Creativity to get original solutions

7. Some of the myths surrounding product sense are

   1. You are born with product sense — The myth exists because an experienced leader with deep Product judgment can often look at something and know what to do, but find it hard to explain.

   2. Product sense is an Art, not Science — To understand how much of it is true, we have to look at the 6 skills of product sense listed above in point no 6. When we analyse that, we realise that barring creativity, everything else is an attainable skills.

   3. People with strong product sense can solve any product problem — Product sense is domain-dependent. Be careful in applying what you know. Cover all of the above before taking the product decisions. Avoid yesterday’s advice – industry, customers, etc. change and so does someone’s product sense in building a particular product

8. How to build product sense (detailed explanation is available in the video and slides, this section is too detailed/nuanced to cover in summary)

   1. Understanding users

      1. Behavioural economics

      2. Data

      3. Talking to customers directly

      4. Segmentation of users to map needs and preferences

   2. Understanding industry

      1. Porter 5 forces and competitive strategy

   3.  Problem framing

      1. Frame the problem from a user’s point of view

      2. 5 Why

      3. Jobs-to-be-done

   4. Tools to design a solution in tech

      1. Coding

      2. Design

   5. Finding analogies

      1. Product case studies from the past/present

      2. Using other products and understanding why they work

   6. Get creative

      1. Start by understanding creativity origins

      2. Simulate and do premortems

      3. Learn more about creativity

Hope this whole session helps you in polishing your product sense. Drop any questions you may have in the comment section.

Have a good weekend,

Deepak

Subscribe now

You can read 50+ posts from the past here - https://www.growth-catalyst.in/archive

Good Morning,

This post is part 2 of the series: Assessing the Effectiveness of PMs and PM Roles. If you haven’t read part 1, you can read it here.

To understand the necessary skills to crack PM interviews, we have to start by looking at what happens in companies across India during PM Interviews.

Knowing What Happens in PM Interviews in India

To understand what happens in the PM interviews in India, we conducted a survey with the readers of Growth Catalyst. The results exceeded my expectations, and this post is being written based on data from 50+ companies, filled by their current or former employees.

We will keep updating this post basis of more responses that we get in the future. Please take the survey if you would like to help the community.

The PM interview process at big global tech (FAANG) is pretty standard across the globe, and there are enough resources a google search away for you to familiarise yourself with them. We aren’t going to cover them here and focus on what happens in the Indian product companies.

From the survey, we can see that the PM interview process across Indian product companies varies to a large extent. Though the data may be slightly off for a single company, they present a high-confidence picture when aggregated.

So let’s start.

Skills

In total, there are 5-6 skills PMs are evaluated for.

1. Product Thinking: Product thinking goes by others names like product design and product sense. You are expected to showcase your product skills in answering

   1. hypothetical scenarios like ‘Design a fridge for a blind person

   2. improvement questions like ‘How will you improve the Uber app?’

2. Problem-Solving: Most of companies do not have a separate problem-solving round, but they rate candidates over it by observing how the candidate solves a problem in analytical, technical, and product sense rounds. Usually, it’s seen the same as with product thinking round.

3. Technical: In the technical round, PMs are assessed on their understanding of technical architecture and why certain choices were made. Questions in this round can include

   1. explaining the current architecture of the app, you are working on

   2. explaining the architecture of a popular app like Youtube or WhatsApp

   3. explaining popular tech concepts like APIs, AI, etc., to a 5-year old

4. Analytical: While assessing analytical skills, you will get questions like

   1. define success metrics of a product

   2. guesstimates like ‘how many balls can fit in a 747?’

   3. define north star metric for a new launch

5. Execution: Like problem-solving, there aren’t separate execution rounds, but candidates are rated around it. Execution means different for different companies, but usually, it’s the ability to prioritize and get things done.

6. Behavioural: Behavioural rounds are conducted by either hiring managers or HRs. Common questions in the behavioural rounds are around

   1. your projects and decisions you made around product, process, and people in those projects

   2. your strengths and weaknesses

   3. your goals/aspirations, and how they fit into the current company you are interviewing for

Only some companies assess all of these skills. Here is the chart showing how many # of rounds were conducted by what % of companies.

We should really look at which ones are more important than others. One way to answer this is by looking at what fraction of companies assesses a particular skill.

If you are a PM who doesn’t come from a software background, you can checkout my book ‘Tech Simplified for PMs and Entrepreneurs’ which has been immensely useful (readers’ word, not mine) in getting them to understand tech well :) 240+ people have rated it 4.5+ on Amazon.

Tech Simplified on Amazon

Skills That Matter

In software development, majority of companies test candidates on Data Structure and Algorithms to hire. But when it comes to product development, there is high variability in the process. Even in FAANG, the process is different for different companies.

So it’s important for an aspiring PM to understand which skills are more likely to be assessed so that they can focus on those skills more. Here is what we got from the survey we conducted.

As you will see, it’s absolute necessary to focus on the product design/problem solving as a skill to crack PM interviews. Analytical and behavioural comes second and third. If you want to prepare very well, I would suggest you prepare for every round. But if you don’t have much time (<6 months), you should focus on product design, analytical, and behavioural.

One may wonder why does the process varies so much across companies. The answer can help you gain insight into psyche of the people recruiting you.

Why Does the Process Varies So Much

The PM is a generalist role, and the roles and responsibilities of a PM varies from company to company. The PM interview process is a reflection of that. Even in the FAANG companies, the requirement differs.

Google and Microsoft build deep tech products like OS, Cloud, Search Engine, etc. and so they want the PMs to have strong technical knowledge.

Amazon, on the other hand, is a business first company, and they don’t have a technical round.

In startups/newer product companies, the interview process is a function of

• skills of the current management team in place

• needs of the product

• quality of PMs available to them basis their brand and funding

For example, technical rounds don’t happen because of one or more of such reasons

• interviewers don’t have a deep technical knowledge

• the management feels their products don’t require strong technical knowledge

• they can’t find good PMs with strong technical expertise

Over time, the process for a company is set by the people they are able to hire, and becomes increasingly difficult to change. But the three factors listed above not only impact what skills are assessed in the interview rounds, but also the depth at which these skills are assessed.

Depth of Skills Assessed

To give you an example, the questions in product thinking can vary from ‘How will you improve Uber’ to ‘Design a robot vacuum cleaner’. The first will test the rigour of your ideas, whereas the second will test many skills including technical, problem-solving, creativity, etc.

A simple analytical question can be around success metrics of a product. But it could become complex when coupled with a problem solving question — ‘how will you reduce returns of Flipkart?’

The depth of skills assessed mostly depends on the interviewer. This is one of hardest areas for an interviewee to judge, and sometimes despite doing everything right, you can fail because the depth at which you are answering doesn’t matches the depth at which your interviewer is evaluating.

So the interview process can get tricky. But don’t worry, we have good resources to prepare well.

Resources to Prepare Well

One of the best resources to prepare well is the book ‘Cracking the PM interview’. You should specifically pay attention to behavioural, estimation, and product design sections covered in the book.

Once you have done decent preparation with the book, you can take some mock interviews with friends/seniors who are already PM. This is a critical stage and you shouldn’t miss it.

Here are some resources for you to explore if you can’t find people to do mock interviews with:

1. https://stellarpeers.com/

2. https://www.pramp.com/pm/uc-product-case-study

3. https://igotanoffer.com/pages/product-manager-mock-interviews (paid)

4. https://www.lewis-lin.com/blog/2018/7/8/lewis-c-lins-slack-community

We would recommend you to try the free mock interviews platforms and not go for a paid platform initially. We haven’t hear very strong reviews for paid platforms yet.

Courses: To Take or Not to Take

Most of the courses for entry-level PMs are expensive, in the range of $500-$3000. For example, upGrad offers a course at $2400, along with others in the same range. The reason these courses are expensive is because:

• These courses have live sessions, and live sessions have a recurring cost to run each cohort.

• The people who can teach product management are highly paid/qualified, and charge a premium to build such courses.

• Assignments evaluated in the course are subjective, and need additional mentors to provide personalised feedback.

• Placement support, if offered, also adds to the cost.

There are couple of red flags you should watch out before taking a course:

1. Time period — To learn the skills mentioned above, such as Product Design, Tech, Analytical, Problem Solving, you need to devote anywhere between 3-12 months, depending on how closely you have worked with the PM function. Live courses usually run at the same pace for everyone and are mostly 2-3 month long. So unless you are already good at many things that a PM does, it is unlikely you would be able to crack the interview. You should definitely skip any course that offers you to be a PM in 4-6 weeks.

2. Syllabus and time taken to cover parts of the syllabus — A case in point here is that you can find courses that spends 1-2 weeks on the topic ‘Tech for PMs’. It would be impossible for someone from a non-tech background (>50% of PMs are from non-tech background) learn enough around tech in a week to clear technical rounds. The good news is that technical round doesn’t happen in many companies so it doesn’t hurt your chances that much. But if they claim to teach you Tech in a week and you end up going in an interview and claim you know it, you are bound to fail the round. Same can be said about analytical round.

So put more weightage to rigour than live learning in the courses. Look at total hours of content, # of assessments, time you need to spent on each of these broad topics in the course.

Once you have done these sanity checks, the final test is the peer reviews. Rather than going by overall placement stats, try to find people on LinkedIN who have taken these courses. Further, narrow down to people who had similar background with you and ask how they found these courses. Were they able to transition? Were they able to get value?

The biggest value an entry-level course can provide you is to get you an interview. That is why I recommend people to take courses after they have done basic preparation from books and taken mock rounds. That way, you have higher chances of doing well in the course, and getting shortlisted.

If You are a Founder/Product Leader Reading This

Here are some steps to follow to build a great PM interview process:

1. Create a list of standard questions for your org — Have a list of questions which interviewers can refer. They don’t have to use these questions, but the list primes interviewers to maintain certain quality in questions.

2. Create sample answers — Sample answers to the questions also helps interviewers see the gaps they themselves might have. You can add best practices in sample answer comments as well.

3. Create an interview panel — Create an interview panel and assign different interviewers different rounds/skills to assess.

4. Ask people to take notes — This should be sacrosanct for all product interviews. Penalise people who don’t do it by doing the next step.

5. Create a review committee — The review committee should read the feedback and discuss to make a decision. It is ideal to have review committee separate from the interview panel (happens in Google), but it may not be feasible for a smaller company. You should definitely have some people in the committee who aren’t part of the interview panel so that they can ask/observe in an unbiased manner. The review committee should act on written interview notes, rather than relying on what Interviewers have to say in this meeting.

If while reading the above pointers you are feeling skeptical, you aren’t alone. There is a reason most of the Indian product companies have a sub-standard product interview process — it’s hard to do it right! But that also means that those who do it right, create a competitive edge.

Hit the like button for the post if it was useful/actionable :)

Thank you for spreading the word and making it happen 🙏

A lot of PMs/ aspiring PMs come from MBA backgrounds where they have prepared very well for the case interviews to crack management consulting jobs.

Many PM interview prep courses also recommend PMs to be good with case studies to sharpen their interview skills, all in good faith. Case interviews and guesstimates help in building a good business and analytical sense.

Case interviews and product interviews are similar because both require you to apply problem-solving skills and arrive at the solution to a problem for a business or an end user. So the skills should be transferrable, right? Assuming this, candidates apply the same skills for problem-solving in product interviews as they did in case interviews. But it ain’t so. And as Mark Twain wrote,

The Difference in Case and Product Interviews

There is a big difference between the case interviews and product interviews, which many people don’t even realize during the interviews. The distinction becomes especially important in the product design round.

• In ‘case interviews’, you are supposed to ask the right data points to solve the problem.

• In product interviews, you are supposed to make intelligent assumptions and narrow them down to the solution. In short, you can’t ask many questions in product interviews.

Sounds counter-intuitive? Let’s dive in.

For the new ones here — 8,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. If you are new here, receive the newsletter in your email by subscribing 👇

Subscribe now

You can read 50+ posts from the past here - https://www.growth-catalyst.in/archive

Let's start with an example. This is what a portion of case interviews looks like (ported from the book “Case in Point”):

Case: Yellow Stuff is a manufacturer that makes industrial cleaning solvents and pesticides. Recently, sales have been declining, mostly due to new EPA guidelines. Further evaluation of sales, both past, and future, indicates that the chemical industry has, and will, continue to grow slowly over the next five to seven years, with 3% annual growth. Management has decided to diversify. While Yellow Stuff wants to keep its chemical business intact, it also wants to enter an industry that has long-term, high-growth potential. Yellow Stuff has hired us to help determine what industry or industries it should enter.

Candidate: So, as I understand it, our client is a chemical manufacturer who wants to diversify outside the chemical industry into a high-growth industry.
Interviewer: That’s right.

Candidate: And you want me to come up with a strategy on how to find the best possible match.
Interviewer: Yes.

Candidate: Besides diversification and profit, are there any other objectives that I should know about?
Interviewer: No.

Candidate: What does the company define as high growth?
Interviewer: 10% a year.

Candidate: Well, the first thing I’d do is obtain a list of all the industries and eliminate the ones that are growing less than 15% or have a potential in the next year of growing less than 15%. How much risk is Yellow Stuff willing to take?
Interviewer: Medium.

As you can see in the example above, asking good questions in case interviews is of high importance. The reason for this is — as a management consultant, you need to ask your clients the right questions to cut through the noise and arrive at the right data points. Clients have the incentive and resources to provide answers to your questions.

As a PM, you need this skill of asking the right questions. However, you have no clients who will bring this answer for you. You have to get the answers yourself. And therefore, there is one more skill you need to do - product intuition or product sense.

Product Sense Matters a Lot

Product sense is all about

• understanding the problem space and its importance for the user

• building the right hypotheses to solve the problem

• using various data points from competitors, industry, and your product to reject and accept hypotheses

As a matter of fact, you would never have enough data to arrive at one final solution. So you rely on your past experience and intuition to make a decision. This is what interviewers are checking for.

So a product interview with Yellow Stuff might look like this:

Case: Mentioned above

Candidate: So, as I understand it, our client is a chemical manufacturer who wants to diversify outside the chemical industry into a high-growth industry.
Interviewer: That’s right.

Candidate: And you want me to come up with a strategy on how to find the best possible match.
Interviewer: Yes.

Candidate: Besides diversification and profit, are there any other objectives that I should know about?

Given that slow growth is a problem it can’t solve due to EPA constraints, the company has to figure out few long-term, high-growth areas. I am assuming revenue growth is the most improtant metric, and they don’t want to focus a lot on profit to start with.
Interviewer: That’s correct.

Candidate: What does the company define as high growth?
As such, growth of 10-12% YoY in most of the sectors is considered good. We can target this number for Yellow Stuff as well. Should I move ahead with that assumption?
Interviewer: Yes

Candidate: Well, the first thing I’d do is obtain a list of all the industries and eliminate the ones that are growing less than 15% or have a potential in the next year of growing less than 15%. How much risk is Yellow Stuff willing to take?

We can take a list of industries and see their growth rates. We can elimate the ones where growth is lower than 10%. One thing to note though is that high growth rates usually come with high risk. High risk can usually be taken by companies which are doing extremely well in their core business, and can afford few setbacks. Yellow Stuff isn’t there, so we should only consider low-to-medium risk businesses. Do you have anything to add there?
Interviewer: No, that sounds reasonable.

As you can see above, the above interview showcases you can make intelligent assumptions, and have a good sense of how to move from the problem space to the solution space on your own. This doesn’t mean you are forbidden to ask good questions in product interviews. Just don’t try to get all answers from the interviewer.

So just as a case interview tests whether you can be a strong management consultant by looking at the quality of your questions and ability to problem solve, a product interview tests your product sense and intuition required to do the job effectively. Sounds good? I would wrap the post with this.

See you in the next one :)

Subscribe now

You can read 50+ posts from the past here - https://www.growth-catalyst.in/archive

Picking which metrics to chase can be tricky.

Some teams pick too many and end up losing focus.

While others pick too few and over-optimize that metric without looking at its side effects.

Some pick vanity metrics that look spectacular on the surface but don't translate to any meaningful business results.

While others pick hard metrics which would be difficult to move within a quarter or so.

Some take months to define the right metrics and go through several analysis-paralysis cycles.

For others, it is more of an afterthought and changes from project to project.

Generally speaking, it’s not advisable to operate on any of the extremes regarding metrics. Three concepts/questions in particular help you define them well:

1. North star metric

2. Countermetrics

3. Sensitivity of metrics

Let’s understand them one by one.

North Star Metric (NSM)

I have written about the north star metric earlier. From Fighting the Evils of Data,

To understand NSM, let’s take Youtube as an example. NSM for Youtube would be total watch time. Let’s see how it helps youtube.

NSM solves a few key problems for the organization:

• Every team at Youtube is focussed on moving this one metric up. The conflicts around priorities and projects can be easily resolved because everyone agrees that this is one thing that matters.

• You have got one indicator of progress, that the whole company understands. So you can explain to everyone how a new initiative helps make that progress.

• NSM holds product teams and other teams accountable. In the case of Youtube, engineers can ask PMs on how a certain feature moves playtime up. This leads to an outcome-oriented culture. The product team isn’t focussed on shipping products, rather creating an impact. Same for other teams.

Now that you know why NSM is important, how does one define the NSM of a product? The secret is asking two questions about the NSM.

1. Does it represent the value created for the customer?

2. Is it a leading indicator of success for the product? The success could be revenue, GMV, or anything else.

The answer to both these questions should be a definite yes.

Let’s test the NSM of Youtube on these two questions. Total playtime represents the value created for the customer because if they aren’t playing videos on Youtube, you can’t be sure if Youtube is adding any value for users.

Total playtime is also a leading indicator of revenue. More playtime —> more ads played —> more revenue for Youtube. Higher revenue makes Youtube successful.

To summarise, you should define the NSM to represent the value created for the consumer and is the leading indicator of the product's success.

North star metric ensures that you aren’t chasing vanity metrics.

Countermetrics

A lot of times, when we chase a metric, it can lead to something else worsening elsewhere. Therefore, NSM should be paired with counter metrics to address unintended consequences.

For example, Airplane manufacturer Airbus wanted to improve the flying experience on its planes by reducing noise in its cabins. Reducing noise is a good goal for them as it improves the customer experience in flight and can lead to more revenues as more people will book Airbus due to improved experience.

But once the noise was removed, Airbus realized that the cabin noise was covering up lots of unwanted sounds — like babies crying and toilets flushing. Its attempt to improve flying actually made the experience worse. 

Countermetrics help in defining a holistic set of metrics avoiding unintended consequences.

Sensitivity of Metrics

The sensitivity of metrics is how easy or hard it is to move metrics on a particular time horizon, like quarterly or annually.

Usually, teams have a good sense of metrics that are too hard or long to move. If you don’t have that sense, here is what you can do:

1. How hard is it?: Mapping initiatives that you believe can move that metric is a good litmus test. If you can come up with the initiatives, that’s a sign that you could move it.

2. How long would it take?: The next step is mapping how long it would take to launch those initiatives. You can add the time, which should give you an estimate of how long it would take to move the metric.

Pick a metric that can move in a quarter, and chase that.

To summarise,

1. Define a north star metric

2. Define counter metrics to that north star metric

3. Check if the metric can move within the intended time horizon

That would be all for this post

Thanks for reading,

Deepak