Before joining the fbFund Rev 2009, the phrases “I think users would love” or “I bet users want” were commonly heard around the NutshellMail office. We were building a product in the way we thought was best, based on our assumptions for what we, and therefore our users, would want. We had a clear vision for our product and were excited to bring it to the world. In early February, we launched our integration with Twitter and Facebook and were ready for a huge influx of traffic. When the anticipated user growth did not materialize as fast as we would have liked, we burned a copy of Field of Dreams and wrote a mean letter to Kevin Costner. “If you build it, they will come” is a crock! We knew we were doing something wrong, but did not know what it was.
It only took a few days at the fbFund to realize that we didn’t know what we didn’t know. However, Dave McClure and the fbFund advisors showed us the way through their mantra of “Metrics, Metrics, Metrics.” The number one lesson from the fbFund is that you have little chance for success without solid metrics. Startups need to be agile, they must iterate quickly to find what works and then replicate. But how can you iterate if you don’t even know what your actual problem is? To be a “metrics-driven business” you must measures everything and then iterate quickly based on quantitative results.
For NutshellMail, this meant changing the way we think and make decisions. We learned that by methodically measuring user behavior, we could quickly understand what our users truly liked, hated or simply did not understand. Instead of making assumptions on what our users wanted, we looked at the stats. The precious information that came from instrumenting metrics across each and every user action allowed us to see exactly how users utilized our features and helped us to measure the effectiveness of any change we made.
But what about simply asking your users what they want? Relying on surveys alone can yield misleading assumptions as there is often a disconnect between what customers say they want and how they actually behave. I am not suggesting that directly surveying users is ineffective. In fact, one of my favorite tools is UserVoice, a customer feedback tool that allows users to request and comment on features. However, metrics take the human factors out of the equation and often provide deeper insights than a survey possibly could. An additional benefit of statistically analyzing user behavior is that the results are more accurate and quantifiable compared to simply asking users what they want through surveys.
So what tools do we use at NutshellMail? First and foremost, Google Analytics is a must to understand traffic patterns, and it is free. I strongly recommend using this in combination with split testing different layouts using Google Website Optimizer. We also use CrazyEgg to analyze what page elements users are clicking on; this is a must if you are testing different page layouts and registration funnels. Of course, we have also layered in a significant amount of customized metrics into our emails and website so that we can identify problem areas quickly. However, if you don’t have the resources or the time to internally instrument your website, a solution like KissMetrics may work great for you.
There are many tools available to you so there is no excuse not to measure. After all, when you assume you not only make an ass of yourself but you significantly reduce your chance of success. Not sure where to start…Check out Dave McClure’s Startup Metrics for Pirates, and you will be well on your way.
Posted by Nirav Batavia and Mark Schmulen of NutshellMail.com, a free service that lets users organize, manage and interact with all their social networks through their favorite email inbox.
