Measuring Long vs Short
As Peter Drucker famously said “If you can’t measure it, you can’t manage it”. Identifying proper success metrics is a critical part of any product strategy. But, to really drive insights from your metrics, measuring alone will not lead to sustained product success. In addition, you need to carefully plan and evaluate your metrics - considering the timespan of your measurement and the causality between metrics over time.
I love when my worlds collide and inspiration comes from entirely unexpected sources. As an example, a few years ago I was supporting my son at his mountain biking race. Downhill mountain biking requires riders to descend technical dirt tracks with a terrifying array of steep drops, mammoth jumps, jagged rock gardens and pinball-worthy tree sections. Since my race craft advice begins and ends with an eye-roll inducing ‘Go Fast and Don’t Crash!’, I calm my nerves at these events by hiking the track and gathering race-side gossip, listening to other racers chew on their strategies for critical course sections.
At this race there was lively debate about the best line choice through one of the wooded sections. Line choice is critical in downhill races; riders will spend much of their practice time ‘sessioning’ - riding a single section over and over to try out different lines to determine the fastest route down. The key decision in this section was whether to go outside or inside a tree near the end. After watching multiple riders trackside, I noticed that most riders opted for the inside line - a shorter path down the trail. But — a few of the more experienced riders chose to go outside.
The winner of the race was Greg Minnaar, the GOAT of downhill racing with 20 plus years of professional experience. After the race Greg was interviewed about his strategy. He explained that most racers only measured their time in the wooded section. Had they continued to measure into the next section, they would have noted the longer term benefit of the outside line. With the outside line, riders maintained more speed going into the next section of the track and this momentum allowed him to carry greater speed through the end of the race. It took Greg’s experience to think beyond an individual section and see the interdependencies between sections of the track. Ahhh - the wisdom of age!
How does this apply to product management? I see a lot of product organizations acting like those less experienced riders, building strategies that myopically focus on short-term goals and measurements. A lot of valuable processes today encourage short term metrics, like agile planning, quarterly business reviews and OKRs. When organizations over-index on short term metrics and goals, they tend to abandon their strategies too early. This may mean pulling investments from product teams, reorganizations, or even layoffs - an all too common remedy for missed metrics these days.
The reality is that the product decisions that you make in enterprise software often take longer than a quarter, 6 months, or even a year to see the impact. Time to impact metrics can be affected by how quickly sales teams can align, time for implementers to absorb product changes, and time for customers to adopt features or change their business processes.
What are better practices to implement to avoid the pitfalls of short term measuring?
Track both short term (monthly, quarterly) and longer term (annual) metrics, with alignment along both organizational structures and along time dimensions.
Be realistic and upfront about time to impact. Often the actions that you take won’t effect change in metrics for several quarters, or even longer.
Measure velocity alongside your core metric to better understand momentum.
Understand, and measure interdependencies between metrics and the underlying strategies that they support. For example, you may not see immediate adoption of a new feature, but the existence of that feature has increased overall customer satisfaction.
And, lastly, always remember that age and experience do have a few advantages!
