Moneyball for SaaS Product Metrics
/“How can you not be romantic about baseball.”
- Brad Pitt, as Oakland A’s manager Billy Bean in the movie Moneyball.
I like baseball, but I love the movie Moneyball. The true story of how Billy Bean and the Oakland A’s changed baseball with a metrics-based approach. Want to win the World Series? Measure getting on base. Want to win with your SaaS product? Measure customers getting on base. Bean and his staff at the A’s applied a simple metric. To win the Series, you needed to win games, to win games, you needed runs. And to get runs, you needed players to get on base.
They had a North Star metric - winning games, and by breaking that down into its essential component (getting on base), they could optimize and experiment. Get the right combination of players who knew how to get on base. Get a hit or a walk, it didn’t matter. Just get on base.
With this framing - they could focus their energy on the leading metric that mattered. They selected players for on-base percentage, then developed strategies that promoted this goal above just swinging for the fence.
Despite having lost their best players at the end 2001, the 2002 A’s used this system to rebuild a team that out-performed 2001, and set the AL winning-streak record. They did this on a shoestring budget and ultimately forever changed how baseball is played.
Here’s how to apply Moneyball to your product metrics. Your first challenge in SaaS product metrics is to identify your North Star metric - what results in a win for your business? Often the answer lies in: what is providing the core value to your customers?
But North Star metrics are typically trailing indicators. They often tell you what happened after it happened, and after the time and activity you truly care about - customers happily using your product - has occurred. Full value for your customer comes at the end of a chain of actions. What are those actions? What is your equivalent to getting on base… the steps that your customers must take with your product to achieve deeper value?
Identify these steps, and you break winning down into the essential components. You get the ability to measure, experiment with, and target each specific action that LEADS to your North Star metric. And measuring and improving customer activation of these component steps moves you into the territory of leading indicators - a much better predictor of progress and improvement for your customers.
Consider an example we implemented at Drip, the ecommerce marketing platform for small businesses. We knew Drip could offer significant value to ecommerce sellers, helping them drive revenue in their storefront.
The North Star for Drip customer product usage was Attributed Revenue - revenue customers gained that could be linked back to marketing they had produced in Drip. This revenue was akin to ‘winning games’ for the customer.
In Drip - there’s a core set of activities customers need to do to drive revenue. They need to connect their store, acquire new prospect, sending marketing campaigns, and build automations for scaling common touchpoints like new customer welcome, post-purchase engagement, etc.
Each of these was the Drip customer ‘getting on base’. We could break down driving revenue to key actions they could take in the platform, then monitor their progress, diagnose blockers, and improve the product (and product marketing) to remove friction. This has several benefits:
‘Getting on base’ activities accelerate your learning. Because you can measure specific leading actions. Wrap metrics around the time it takes a customer to send their first marketing email for example.
You can explore and experiment with specific actions. What is blocking that specific task? In the email example - is it content creation? Is it uncertainty on the value of the email campaign versus other tasks? Is the UX clunky?
Fine grained actions make it easier to explore specific challenges of your customers in 1:1 conversations and other user research activities. And they make it easier to experiment with different solutions, even ones that won’t scale at first, to improve the action.
As you get comfortable with what specific actions comprise ‘getting on base’ for your product and customer - you can explore things like - does the order matter? Is there a first base? And is there a pathway through these actions most commonly chosen by the best customer?
Moneyball for SaaS is a natural compliment to product-led-growth and is especially valuable in onboarding and early stage product usage. At Drip - it was key to us improving our early-stage retention which bends the churn curve. And most important - it gave an early indicator of progress - if North Star attainment is measured in weeks, ‘getting on base’ steps could be measured in days or sometimes… hours.
“How can you not be romantic about SaaS?”. Ok maybe that’s pushing it as there’s not typically sun and hot dogs involved. But at least there’s a good chance of beer.
And it’s still a lot of fun to play the game.
CODA: Like Bean had Sabermetrics and Bill James to draw from, Drip had the benefit of working with Greg Daines and Jeff Moss at Client Velocity. Their approach to churn curve bending ultimately influenced me to implement ‘Moneyball’ at Drip.