Co-authored with Cori Widen, researcher and product marketing extraordinaire.
Authors’ note: This guide was written before COVID, when in-person meetups were possible. But nearly all of the principles here apply to virtual meetups as well.
Why is spending a day with people from your target market such a potential game-changer?
If you’re a product manager, a marketing manager, or in a related field, you’re constantly being called upon to make critical judgement calls. You have to define your product roadmap, create brand guidelines, and a whole school of other tasks that can make or break the next month, quarter, year, and so on.
Spending time with users, asking the right questions, and analyzing their experiences helps you to make user-centered judgement calls. When you do this, your company’s product development vision relies less on internal assumptions of “what we want to build” (i.e. a risky proposition) and more on the essence and practicalities of the people you’re actually building products for.
Because we’ve seen the power of such day-long events to help teams unearth pivotal, strategic insights, we’re sharing our experiences from organizing some particularly eye-opening sessions. This guide doesn’t include every single detail you may ever need to know, but our goal is to give you enough information and motivation to organize your first transformative experience.
We know, from our own journeys, that once you feel the inevitable high, and the fallout of ideas and insights after a day with your users — the rest will follow.
Without further ado, here’s how to do it.
Step 1: Define scope
Step one is for stakeholders to get together and define the questions that they’re trying to answer. These aren’t the questions that you’re going to ask your users; this is a list of the questions you’re asking yourselves.
Questions are crucial: you have to be asking the right questions and checking that what you’re trying to find out is really what you need to know to help inform your product vision, marketing plan, or whatever you’re working on.
Some examples of questions and starting points that will yield you less data, or less reliable data are:
“Will people buy X?”
“How can we get people to do X?”
“Let’s hold an event to let people know they can now do X with the product.”
“Let’s create an event to launch this new product with testers.”
Some examples of questions and starting points that will yield more data, or more reliable data:
“How do members perceive our current pricing model?”
“Why aren’t people doing X with our platform?”
“How will people react to these new tools we are building?”
“What are the challenges that users experience when doing x?”
Scope questions can be direct product insights (for example, those that relate to potential features) or bigger open questions that help you understand the inner workings of the people and processes that you’re building for. Good examples are: “What are these people trying to achieve?” “How do they define success?” or “What frustrations do they experience?”
You’ll likely have multiple scope questions, and that’s a good thing. Start out with a long list and then combine and prioritize so that you know where to place emphasis when you’re planning the day.
Once you’re satisfied with your scope questions, see if you can group them based on topic. This will make it easier to think about how to structure your sessions throughout the day. For example, you may find that your questions fit into groups like: experiences with our competitors; sentiments about Y; challenges with X, and so on.
Photo by Curtis MacNewton
Step 2: Recruit participants
First, think carefully about which type of people might best be able to shed light on your main questions. If you have a large user base, you will find lots of opportunities to get quite specific: you can reach out to people who use the product in a certain way, people who are beginners, or people who are dissatisfied for whatever reason.
Most often when doing qualitative research, dividing up people by a common motivation yields the most insights, so the time spent on the decision of who to recruit is time well spent. You may even choose to divide the day or divide your workshops between two different groups of people. Think carefully about the dynamics that are likely to develop between participants, and work from there.
If you’re looking to bring in people who are already familiar with what you do, it will of course be easier if you’ve kept a relationship going with a possible pool of relevant participants, such as via a dedicated Facebook group for customers to give feedback about the product. If you haven’t done this, then now is probably a good time to begin.
In your invitation or call for participants, provide the appropriate incentives. For example, incentives for power users can be gifts or gift certificates that speak to users’ interests and/or align with the company mission. If testers are not familiar with the product, you may want to provide cash. If you’re concerned that there might not be enough interest in joining, you may want to offer a giveaway via your promotion channels (eg. one lucky participant wins a trip to a travel destination).
It helps to be clear up front about which logistical costs you will cover and which you will not: eg. meals or snacks included, travel fares to the event, etc. All this information will help participants plan and commit to attending. And, if you have done your organizing far in advance, be sure to send participants reminders a couple of days before the event as well.
You can also pad the attendance by overbooking up to about 25% of the desired number of participants to make sure you have enough in the likely event that some people cancel.
Photo by Tim Gouw
Step 3: Plan the day
Plan the entrance and your space: set up the space with chairs facing each other, set out name tags, drinks, etc. Make sure that your seating arrangements are such that your team is intermingled with participants to keep the feeling casual rather than confrontational.
Start the day with context. You won’t share your scope questions and you’ll want to say as little as possible about your actual research so that you’re not already inviting the answers you want to hear subconsciously. However, you do want to frame the day. A great formula for opening the day is your company mission and a general statement about what you’ll be discussing. For example, “At Company X, we empower small business owners to do Y, and today we want to learn more about how you do Y so that we can succeed even further in meeting your needs.” This gives enough information and tells people what to expect, but avoids the research version of TMI.
Start with introductions and an icebreaker. Everyone feeling comfortable around each other is going to affect how much, and what, you hear throughout the day.
Play games! Add games or structure sessions as a game wherever possible. These keep the energy level high, foster friendly collaboration and sharing, and help participants feel less under pressure to “perform” or “deliver”. Some ideas for collaborative games are treasure hunts/scavenger hunts, role play games, card games, all types of variations of charades, as well as relay “races” where participants need to complete tasks with a clock ticking.
A great example: the research team at the drafting software company AutoCAD created an escape room for their event. Participants in the closed room were shown a mini action video (complete with costuming and over-the-top music!) which told them about a local crime. Then, participants needed to use a drafting app to help the police solve the mystery of how the burglar had escaped from the scene of the robbery, and to find his hiding place. With participants in “play mode” rather than feeling like lab rats, they were able to use the product far more naturally, giving the research team valuable insights about how they’d use the product in real life.
Get tactile: crayons, paints, puzzles, string. Structured activities where people are using their hands (rather than just talking) aren’t just more fun, they get people talking more freely.
This is an example that we worked on together at a day-long event for the DIY community platform, Hometalk. We handed out markers, paints and paper for the participants to collage together their dream home page. This activity had a number of benefits: Firstly, it was on-point in terms of subject matter, since this was a community of crafters and makers. Secondly, it fostered camaraderie and helped people express themselves more openly. And lastly, when we took everybody’s creation and displayed it for discussion, we found that people had made seemingly simple graphical decisions that actually had complex ideas behind them. If we hadn’t asked participants to draw or make things — they wouldn’t have brought up these ideas verbally.
Card-sorting or word games are always fun, and are a versatile mainstay of qualitative data gathering: write different concepts, ideas, or features on cards and have people sort them according to priorities. Or, write different words on cards and have people sort them into groups. This can be used to answer very varied questions: use this method to see which words (ie emotions) people most associate with your brand or product. Or use it to see which features may be useful. You can also use it to see which functionalities should be grouped together in your product’s architecture or navigation system.
Plan breaks (including meal breaks) carefully. Reduce chaos by having food delivered or stocking up the fridge beforehand. Understand that these days are exhausting for everyone, and you don’t want people to lose steam.
Structure a conversation session (or two) around a short list of carefully crafted questions. Two or three open-ended, pointed questions that cut straight to the chase. Go around the room and let each person share an answer. If you are worried about groupthink, you can also give all the participants a few minutes to jot down their answers first, before everyone shares what they wrote. This may be a good forum to ask users about their central challenges, sentiments, and other information that becomes more useful when a discussion ensues.
Ideally, you will want to make sure that a staff member is recording all of the events in an unobtrusive manner and that the participants have been put at ease with this. Having these recordings later on will allow you to bring insights to the rest of your team, and possibly provide a reference point if you are unsure about something and want to go back to check.
Challenging consensus: an example
An absolute must for avoiding groupthink during your sessions, and a general mantra for the day is: challenge consensus! If you hear the exact same response from most participants, ask clarifying questions to try and dig up the nuances. Here’s an example of how this played out at a focus group where we were exploring how to help people complete home improvement projects:
Facilitator: What’s the most difficult part of planning a DIY project?
Participant A, answers quickly: getting the materials! You often have to go all over the place. It’s either that or waiting for an online order, but it’s not usually in one place.
Participant B: Ugh, yes, it’s so annoying.
Participant C: Also, when you order online, you already know that when something comes, you’re not going to like it as much as you thought you would!
:::people laugh:::
Participant D: Yeah, and when you go to Home Depot, you know that you’ll have to walk a full marathon before you find the thing that you do like!
Facilitator: Getting the materials sounds like a key part of the planning process. What else is involved in planning a project once you have an idea?
Participant E: Well, there is the whole thing about making sure you know what you’re doing before you start doing it. Like if you’re using a new tool, a different painting technique, or something like that.
Facilitator: Are there any challenges when it comes to making sure you have the knowledge you need to do a project?
Participant A: Well, searching for the videos and tutorials and making sure that it’s quality and easy-to-understand isn’t always easy.
Participant B: Yeah, and also, how do you really know when you’ve understood enough? At some point, you just have to say — ok, I’m going for it.
In this example, had we not challenged consensus, we could have just walked away saying that finding materials stands out as the main pain point in planning a DIY project. But sometimes, the conversation sways where the first person who talks takes it, and the group needs additional prodding to get to other interesting points.
In all your activities, create a safe space for sharing honestly, and for dealing with disagreements. Never confront, as you will not get honest answers when anyone is on the defensive. Remember that in the context of participants sharing their views, all views are valid. Most of the time the only thing you should be doing when someone shares a point of view is asking an open ended “hmm, why is that?” or “can you elaborate on that?” If participants disagree amongst themselves, or get fixated on one idea, simply try and move the conversation onward to other topics.
So many questions!
As you’re planning your sessions and writing down the questions you’ll ask your participants, keep in mind that there are some ground rules for asking questions. An entire book could be written about this one topic, but we’ll give you a few pointers here for some general guidance:
Avoid leading questions: oftentimes, without thinking, we signal to participants the answers we want to hear and since people are generally aiming to please, they tell us what we want to hear. Try to ask questions in a neutral way to avoid this. Here’s an example of a leading question: Is finding materials for a DIY project one of the most stressful parts of the process? Instead, you’d want to keep the question open and general, like in our earlier example: What’s the most difficult aspect of doing a DIY project?
Don’t ask users to predict their own behavior. Spoiler: people are extraordinarily inept at predicting their future actions, because everything is tied up in how we want to see ourselves and how we want others to see us. Spare yourself the act of barking up the wrong tree by avoiding questions like “If we built X feature, would you pay extra for it?” and instead, ask the right peripheral questions so that you can get the right insights.
Be open to surprises and detours. You’re not here to confirm your hypotheses; you’re here to learn. It’s inevitable that throughout the day, you’ll hear things from your participants that are surprising and far from your original expectations. See this as a success, not a failure, and don’t be afraid to ask questions you hadn’t planned in order to fully explore these new potential insights.
Photo by Alexis Brown
Step 4: Process what you’ve learned
When it comes to qualitative analysis, it often takes a variety of perspectives to get to the heart of the issues that we care the most about, in a way that is the least subjective.
(Note: If you invited stakeholders to join you for the day, but they won’t be helping out with the analysis, we suggest giving them a heads up about when you will be sharing your insights, and encourage them to refrain from drawing actionable conclusions until then.)
Here’s how we suggest debriefing:
You’ll want to have a team session for everyone who was at the event to go through, session by session, and share their impressions.
Doing this as soon as possible is key (fatigue-permitting!). The best model we’ve found is having an off-site discussion the day after. It’s best to do this outside of the office where you have the time and space to really delve into what you heard.
Allow time for debates. Maybe you took away an insight that, at first glance, seems to conflict with a colleague. Talking it through almost always leads you to a more nuanced insight than you would have had on your own.
Have a visual: someone should be writing key insights and takeaways (we love post-it notes) as you move through the analysis of the day. Later, these will be categorized for easier consumption and further reflection.
Photo by You X Ventures
Step 5: Share insights
For maximum impact, be creative and be quick — don’t sit on your learnings very long before sharing. Anything goes here. Some effective ways to bring insights to team members, stakeholders, or company leadership are:
Video clips or a montage of the day’s most interesting moments (including 360 degree go-pro videos!)
A slideshow or presentation that has photos interspersed with insights
Recordings or even a podcast. One creative researcher I met, even recorded a podcast about her event and what was discovered, which was then shared with the entire company to listen to.
Ask the team some of the questions that were posed to the day’s participants, to get them in their shoes, before presenting insights.
Q&A after sharing your insights: teams will have a lot of questions. Remember that “we can’t be sure” or “this wasn’t a part of our research” are legitimate phrases to use. If you don’t use them, you could be inadvertently presenting your assumptions rather than your research insights.
A lunch-and-learn model is a great way to loop in additional colleagues who may not be direct stakeholders but who would benefit from knowing more about your target market.
Our experience has been that even if you find a lack of enthusiasm or skepticism before you embark on your first day with users, sharing your insights with stakeholders generally results in requests for more research and people will ask to join in on the fun, next time around.
When writing this guide, we felt that each section could have been its own post (and maybe we’ll be writing those!) but we hope that we’ve given enough advice and inspiration for you to say: Ok, l can try this!
We’ve seen that even small steps toward regular qualitative research within a company can have a large impact, and that the process improves over time, so you reap more benefits as you go.
Do you want to organize a research event? Or have an experience to share? Reach out and let us know!