How we got to the metrics to help validate our early-stage startup.
When you join your first startup, there’ll be days when you think you’re just rubbish at your job.
Despite working hard, putting in the hours, and applying everything you thought you knew, you’ll still find yourself asking: ‘why did I bother getting out of bed this morning?’
You’ll also become easily frustrated by the little things.
Changing course, not letting things bed in, always struggling to explain what you do as a job to your Mum.
The thing is, this is the inevitable journey you’ll go through when validating a startup and working out which metrics you need to focus your efforts. Because if you don’t use the right metrics, you’ll be chasing the wrong thing.
What are common startup metrics and frameworks?
Many people would say that metrics aren't the most sexy of topics. But, I kinda like them. I like the visibility it can give a business; showing how everyone’s work is helping move the business forward.
I’ve been lucky to work for a few startups at various stages, and have encountered some lovely frameworks. Actually, make that a lot of frameworks.
But that’s the problem, there’s many available… which do you choose?
Frameworks I have used include:
The Lean Canvas - Maurya
Pirate metrics - McClure
Revenue metrics
Long and short funnel metrics (tofu, mofu, bofu, CTRs, ORs)
Subscription metrics (MRR, churn)
SaaS metrics - which are kinda a mixture of the above.
And here are some of the factors that will impact the metrics you’ll use:
The stage the startup is at
The industry and model you operate
Bias from previous business models and experiences
Hippos.
So that’s the scene, that’s what we are working with. Broadly speaking.
Now, I am going to talk about some of the actual metrics we used to help validate our early-stage startup.
(Before we jump in: when I refer to product-market fit, I use the term loosely. I don’t believe that you can truly achieve this; markets change, how a customer derives value changes).
The biggest mistake when using startup metrics: misunderstanding the stage the startup actually is at.
When joining an early-stage startup in your first marketing role, throw out the idea that you’re going to be carrying out marketing like you did at your previous job.
If the startup you are operating in is at the early stage - and by this I mean the founder has identified a real problem that needs addressing, and you're there to validate it - your title won’t be marketing.
Marketing, as you know it, doesn’t exist in the guise you know it to be. Yet. Chances are, you’ll be carrying out customer discovery to help work out if people will pay for the service/product your founder is proposing.
It happened when I joined Careercake.
I joined as employee number one as a digital marketer. Very quickly, however, I realised the metrics I was trying to apply to the business didn’t fit. Nothing seemed to be working. Yes, I had people subscribing and engaging with the activity I was carrying out, but it wasn’t turning into the growth I expected.
The mistake I had made was thinking the startup was much farther developed than I first thought.
I realised after a period of time that the business wasn’t at that stage yet.
It was still in the validation stage.
And when you’re at the validation stage, the KPIs you use will be completely different to those you’d use at the customer demand stage. Massively.
What stage your startup is at will determine what you do, and the metrics you use.
Metrics we used to help validate our early-stage startup as a platform for growth.
We spoke to a lot of people who all had their own take on which metrics we should use. Some insights were great, don’t get me wrong, but sometimes I felt we were shoehorned into particular frameworks that weren’t right… because the business wasn’t at that point the expert/advisor thought we were at.
I really like Croll & Yoskovitz’s analytical framework, and we applied it as it made sense for us. Using this framework, therefore, meant Careercake at this point had ticked the following:
“I’ve found a real, poorly met need a reachable market faces”
The bit we were now tackling was:
“I’ve figured out how to solve the problem in a way they will accept and pay for”
The activity we would need to carry out next was to see if the value proposition ‘works’, and understanding how to activate and retain a customer on our MVP.
We’d already secured a handful of clients in our core segments. Now it was about opening up the segment with a value proposition we hoped they’d pay for and keep using.
And then learning the metrics / KPIs /OKRs - whatever you want to call them - that made sense to us.
Here’s how it went down.
Is there a need for this service/product?
The problem:
A lack of careers advice that young people actually wanted to watch and, more importantly, do something about their career advancement afterwards.
The realisation:
Our CEO had recognised the need for careers advice that was delivered in the way our end user wanted (video) and talked about the topics they actually faced, and not the traditional concepts.
Okay, so we’ve got a problem that we think needs addressing.
Is it strong / painful enough that people will part with their cash to solve, and what are the indicators of ‘success’ I needed to look out for?
An MVP that was created and launched to test a few things, including:
will b2c audiences pay for this type of advice?
which core video topics would need to be included to enter the b2b market?
and, of the community already built, how many - if any - would pay?
The plan.
We looked at the potential customer segments we had and agreed to spend five months targeting a specific industry with the aim of learning things such as:
- does the value proposition actually make sense?
- what number of courses or which type of courses does someone need to watch to become a paying customer?
- does the proposition make sense for new b2b segments we wish to target?
Some the frameworks we used to help validate our startup,
based on Lean Analytics (Croll + Yoskovitz)
What we learned…
We were using the right early stage metrics, but in the wrong context.
People use a service like Careercake at what’s known as ‘the point of pain’. For example, let’s say someone has been made redundant and they’ve not been active in the job hunting market. They need to know how to position their skills or use LinkedIn. And they need to know quickly.
I’ve spoken about it previously. When we spoke about engagement of Careercake content we set ourselves big targets: ‘we’ll be the Netflix of careers!’ ‘People will binge our content and keep coming back!’
It’s just not how it works. Once someone had landed a job, passed their probationary period, and felt a little more confident, they didn’t need our reactive service. The next time they’d need it was when they faced another ‘pain point’ in their career, such as running their first meeting or creating an elevator pitch.
In this case you realise that you don’t use a service like ours in the same way as you’d use say, Spotify or iPlayer.
Metrics used:
Daily and monthly active users, answers to survey questions upon sign up, questions if they didn’t renew. All were then aligned around seasonality and events, such as when people switch jobs or when students get ready to graduate. We looked at the metrics a platform would use but drilled further into the context.
Tools used:
Focus groups to understand usage patterns, Google Analytics, HotJar.
Learning the golden set of actions that go on to equal a paying customer.
We’ve got five different b2c customer personas who sign up and use Careercake. A handful of them are profitable, others are subsidised by these converting segments into paying customers.
For us to get to this realisation, we had to experiment and learn which source would bring in the right type of customers, and which combination of courses watched and engagement plotted over 2 days, would lead to a paying customer.
Tools used:
Profitwell, Chargebee, Vimeo, Google Analytics, Intercom.
Metrics:
SaaS metrics - mainly, sign up rates by segment, conversion to paid, LTV, churn.
We were biased around our pricing assumptions.
We had a community who were used to watching Careercake for free. When testing to understand at what level pricing sensitivity kicked in, we realised very quickly that there was a customer segment who were not prepared to pay an increase, despite us adding new titles and features.
Despite us thinking that they would pay, despite them telling us in early customer interviews that they would.
Metrics used:
Threshold of respondents who answered that the price was expensive, redemption of loyalty coupons.
Tools used:
Email newsletter (dotdigital) and surveys.
Oh, and they hated the brand.
Okay, yes, this isn’t a metric per say, but it was something that is still really important to note.
On camera, Careercake content is authentic, honest, colloquial, and real. But the design of the old site, the overall feel, well… it was designed, we felt, to keep the people in suits happy. (Full disclosure, our investors never asked for this).
It turned out, the end user didn’t like our brand one bit":
In fact, one person referred to it as ‘a bit meh!’
Genuine personal career highlight, that.
In terms of how the brand was received by potential new b2b customers, it was the same. These potential customers were drawn to the content of Careercake, which meant boring, traditional branding would not do if we wanted to enter markets with the value proposition around increasing engagement rates.
Tools used:
Mystery shopping, 10 customer interviews a month via Skype/face to face, online polls and exit surveys, 5 b2b prospect interviews via referrals.
Metrics used:
Whilst this was a qualitative piece, the metrics used were around ‘do they have the need for this type of service in the first place?’ And, when given a demo, ‘did they actually sign up, activate and watch a minimum of three courses?’
What happened next?
Well, we launched MVP 2.0 using this feedback. Overall, the increments were positive - we saw more people signing up, and within three months landed our first enterprise client. (Yay!)
All because we took the time to work out the metrics to help validate our early-stage startup that made sense for us.
But then came the next stage, validating further the b2b markets, looking at what features they do and do not use and ultimately, what can we do to increase LTV, reduce churn within key segments and how do we grow.
That’s all for another article ;)
Some useful resources validating your startup include:
11 Metrics Every SaaS Company Should Care About
The Stickiest, Most Addictive, Most Engaging, and Fastest-Growing Social Apps—and How to Measure Them
What Do I Do Now? The Startup Lifecycle
The Startup Marketing Template for one-person marketing teams.
