The basic principle behind data driven design is that digital products can be better conceived, planned, and implemented when they are informed by real-life feedback. This way of working has its original roots in the scientific method, where all preconceived ideas about how something should work, or even whether it should exist at all, are hypotheses to be tested and proven. In order to do this, a simple version of the product is built as soon as possible (ideally within hours or days), and tested with real-world situations or simulations. This is the methodology used by nearly all product managers and product designers today.
Throughout the lifecycle of a product or service, data-informed design helps teams to gain insights that can help minimise risk, reach product market fit, reduce costs, and identify market opportunities. However, data driven design isn’t a magic bullet to success: it simply lessens the probability of failure. It can be easy to forget that building any new product or service is always a risky proposition, speculative in nature, and fraught with unexpected challenges at every single turn. It’s also easy for teams to get caught up in metrics that don’t really matter, to misinterpret results, or to create a tunnel-vision outlook for themselves where they fail to recognize when the writing’s on the wall. Here’s what you can do to avoid the pitfalls:
Build your pipelines constantly
It’s never enough to rely on the same expected data pipelines. Building new ways to measure and new ways to investigate is a surefire way to reduce the chances that you’re missing out on important information.
Challenge assumed constraints
If you’ve tried a number of new things to reach a certain goal, and none of them seem to have any measurable effect, you’ll need to get creative. Either you’re designing the wrong thing, looking at the wrong metric, or have mistaken assumptions about what matters to customers. The iterative process demands of us to constantly question whether the things we assume to be true about the world - actually are.
Be humble
Have enough humility to recognize when the data is showing you that something isn’t working, and you need to make difficult choices about how to proceed. Often we miss the warning signs that what we are building is on track to fail, until it’s too late.
Understand the spectrum of experience
When looking at your numbers, a fact that can be overlooked is that today’s users often are served very different experiences depending on their device, location, or (In the case of machine learning products) very specific person profiles. It’s always important to try and investigate why certain experiences might be doing better than others.
Remember the OMTM
A large quantity of user feedback data isn’t necessarily going to help - you need to know whether the metrics you’re looking at are actually important. The one metric that matters” (OMTM) is the single number that matters most to your current growth. It allows you to focus in moments of overload, and remember what’s really important.
Take the long view
Remember that a good pivot, based on data discoveries, is one that keeps you in the game, rather than beats the competition. Truly innovative discoveries come when we compete against ourselves.
So get out there and get curious! Soon you’ll be finding those golden data nuggets FTW.