Product Metrics: North Star Metrics, Counter-metrics and Sensitivity

Picking Product Metrics to Chase

For the new ones here — 8,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. If you are new here, receive the newsletter in your email by subscribing 👇

You can read 50+ posts from the past here - https://www.growth-catalyst.in/archive

---

Picking which metrics to chase can be tricky.

Some teams pick too many and end up losing focus.

While others pick too few and over-optimize that metric without looking at its side effects.

Some pick vanity metrics that look spectacular on the surface but don't translate to any meaningful business results.

While others pick hard metrics which would be difficult to move within a quarter or so.

Some take months to define the right metrics and go through several analysis-paralysis cycles.

For others, it is more of an afterthought and changes from project to project.

Generally speaking, it’s not advisable to operate on any of the extremes regarding metrics. Three concepts/questions in particular help you define them well:

1. North star metric

2. Countermetrics

3. Sensitivity of metrics

Let’s understand them one by one.

North Star Metric (NSM)

I have written about the north star metric earlier. From Fighting the Evils of Data,

To understand NSM, let’s take Youtube as an example. NSM for Youtube would be total watch time. Let’s see how it helps youtube.

NSM solves a few key problems for the organization:

• Every team at Youtube is focussed on moving this one metric up. The conflicts around priorities and projects can be easily resolved because everyone agrees that this is one thing that matters.

• You have got one indicator of progress, that the whole company understands. So you can explain to everyone how a new initiative helps make that progress.

• NSM holds product teams and other teams accountable. In the case of Youtube, engineers can ask PMs on how a certain feature moves playtime up. This leads to an outcome-oriented culture. The product team isn’t focussed on shipping products, rather creating an impact. Same for other teams.

Now that you know why NSM is important, how does one define the NSM of a product? The secret is asking two questions about the NSM.

1. Does it represent the value created for the customer?

2. Is it a leading indicator of success for the product? The success could be revenue, GMV, or anything else.

The answer to both these questions should be a definite yes.

Let’s test the NSM of Youtube on these two questions. Total playtime represents the value created for the customer because if they aren’t playing videos on Youtube, you can’t be sure if Youtube is adding any value for users.

Total playtime is also a leading indicator of revenue. More playtime —> more ads played —> more revenue for Youtube. Higher revenue makes Youtube successful.

To summarise, you should define the NSM to represent the value created for the consumer and is the leading indicator of the product's success.

North star metric ensures that you aren’t chasing vanity metrics.

Countermetrics

A lot of times, when we chase a metric, it can lead to something else worsening elsewhere. Therefore, NSM should be paired with counter metrics to address unintended consequences.

For example, Airplane manufacturer Airbus wanted to improve the flying experience on its planes by reducing noise in its cabins. Reducing noise is a good goal for them as it improves the customer experience in flight and can lead to more revenues as more people will book Airbus due to improved experience.

But once the noise was removed, Airbus realized that the cabin noise was covering up lots of unwanted sounds — like babies crying and toilets flushing. Its attempt to improve flying actually made the experience worse. 

Countermetrics help in defining a holistic set of metrics avoiding unintended consequences.

Sensitivity of Metrics

The sensitivity of metrics is how easy or hard it is to move metrics on a particular time horizon, like quarterly or annually.

Usually, teams have a good sense of metrics that are too hard or long to move. If you don’t have that sense, here is what you can do:

1. How hard is it?: Mapping initiatives that you believe can move that metric is a good litmus test. If you can come up with the initiatives, that’s a sign that you could move it.

2. How long would it take?: The next step is mapping how long it would take to launch those initiatives. You can add the time, which should give you an estimate of how long it would take to move the metric.

Pick a metric that can move in a quarter, and chase that.

---

To summarise,

1. Define a north star metric

2. Define counter metrics to that north star metric

3. Check if the metric can move within the intended time horizon

That would be all for this post

Thanks for reading,

Deepak