Founders Should Test The Hypotheses That Matter Most
It is a well-known mantra that startups are experimentation machines. What is less well-understood is the process for determining which experiments to run and how to sequence those experiments. Given that founders have a discrete envelope of time and money before they need to produce positive results, experimentation selection is one of the most critical factors in startup success.
In my Harvard Business School class on pre-product market fit startups (Launching Technology Ventures — or LTV), we talk a great deal about business models. In particular, we deconstruct the business models of startups into four main components: (1) consumer value proposition; (2) go to market; (3) profit formula (which incorporates market size and unit economics); and (4) the product, technology and operations.
Taking a page from Tom Eisenmann and Eric Ries’ writings on hypothesis-driven entrepreneurship, we emphasize the importance of formulating hypotheses and running tests. Much has been written about the tactics behind running tests (e.g., constructing a minimum viable product, or MVP) and the importance of pivots in response to test results, but what many entrepreneurs miss is that the best lean tactics do not make up for a bad strategy. Test selection is all about strategic choices.
Entrepreneurs need to be thoughtful and disciplined regarding test selection, design, and prioritization. In short, entrepreneurs need to analyze their business model critically and select the tests that matter the most at each phase of their journey.
For example, we study the online search startup Aardvark, who executed Ries’ “Lean Startup” tactics perfectly. Unfortunately, the company failed because they never developed a consumer value proposition that users found compelling enough to continue using after an initial trial. In short, the company invested heavily in testing its product machine (i.e., building out the product, technology, and operations) but didn’t rigorously test the consumer value proposition: who is going to value this service and use it on a continuous basis and why?
We also study Classpass, a fitness company that has succeeded tremendously but almost slammed into the wall — growing unsustainably with poor unit economics — because they failed to run business model experiments early on in their subscription offering. In contrast to Aardvark, they did a great job rigorously testing the value proposition but then rushed to scale prematurely when they hit initial product-market fit before testing and deeply understanding their pricing, packaging, churn and ultimately, lifetime value vs. customer acquisition costs.
Aardvark and Classpass were started by brilliant and experienced founders, had very successful investors and struck a chord with a large customer base. Hindsight is 20–20 and thus a useful analytical tool that should be applied with humility. That said, both companies could have seen more success if they had more rigorously prioritized the most important component of their business model to test and constructed narrow tests that required more modest capital and time.
Generally speaking, I find that the consumer value proposition tests are the most important initial tests to focus on. Once you nail the consumer value proposition, the go to market plan can flow and once both of those components are locked down, the profit formula can be tested. That said, each of the four business model components are tightly intertwined with dependencies and so selecting which ones to focus on and in which order is a delicate decision that varies from startup to startup.
Three questions to ask yourself to help determine which test to prioritize:
- Which of the four business model components is the most surprising or controversial and thus do I need to focus on right now? Focusing on that one component, what is the essential hypothesis that I need to test to prove out the power of that business model component?
- What is the key milestone that I need to achieve — either proof points or learnings — to lead to a valuation inflection point and make a follow-on financing easier to raise? Which aspect of my plan are investors challenging the most and, thus, requires the most data to refute?
- Where does the greatest risk exist in my business model and what does the flow of dependencies look like? In other words, if I can unlock this one thing, will everything else flow more easily?
Waze founder Noah Bardin wrote an excellent article on the essence of a founder’s job, which is to maniacally focus the company on the Most Important Thing (MIT). I would modify this view slightly and argue that in the pre-product market fit stage of a startup’s life, the founder’s job is to maniacally focus the company on the Most Important Test (MIT).
📝 Read this story later in Journal.
🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: Get the noteworthy newsletter >
