I'm a huge fan of hack tests. For example, let's say you believe that gardeners will buy more gardening equipment if you create a content-focused startup where the content's goal is to drive e-commerce. Instead of spending months building a site/app, setup a MailChimp account today and start sending emails tomorrow with links to gardening equipment on other sites. What's the open rate? What's the click rate? Did people buy things? Are people sharing your email with others?
Nine times out of ten the founders are unhappy with these results, but something this small can be changed quickly or thrown away for the next hack test.
However this type of quick/cheap/agile testing can be a siren song because there's a hidden disadvantage...time/effort. Much of the calories that a founder saves in building product is taken-up by the effort required to manually test because - with hack tests - you have to spend way more time doing manual work to offset product and then there's all the time talking with customers about their thoughts on the core elements that you've put in front of them.
I'm still a huge believe in hack tests, but I wouldn't assume that 90% of the time/effort is saved.
I was reminded of this when I heard Sam Altman, who was at the time of this podcast the President of Y Combinator, say that in both his startup and YC they encourage founding teams to be ruthless about prioritization to “say ‘no’ to good ideas to save firepower for great ones.”
Here's how I think about the friction...
Good founders are able to "walk through walls" to make sure that their products make it into the world, but time/money/resources are very scarce, so simply dumb/blind passion/energy isn't enough.
Almost all ideas in the beginning aren't exactly right, so founders have to try lots of things.
But trying things - either manually or by building product - takes lots of time/energy/resources.
So how does a founder reconcile trying lots of things with focusing limited resources?
By saying "no" to most things.
I would encourage founder/CEOs to research lots of things and be open to lots of new ideas, but decide to only focus on a few. Then decide the steps to hack test each of them with the most important outcome defined ahead of time (so you know the single thing you are measuring in the test). Then pay particular attention to surprising results.
Sidenote: If you enjoyed this post, you might like this one as well.
Get Right to the Lesson
I’d recommend listening to the entire thing, but to get right to the point go to minute 4:19 of this podcast.
Thanks to these folks for helping us all learn faster
Sam Altman (@sama) of Y Combinator (@ycombinator)
Andrew Warner (@AndrewWarner) of Mixergy (@Mixergy)
Please let me and others know what you think about this topic
Email me privately at email@example.com or let's discuss publicly at @davempayne.