3 Homegrown User Research Techniques & How To Do Them Right
Thoughts on strategy delivered to your inbox.
The worst aspect of “user research” is the coldness of its name. The people—the human beings—that your organization serves aren’t “users.” They’re just as complex as you are, just as prone to whims and distractions and prejudices and shifting interests. Your constituents aren’t a single, homogenous bloc.
And because your constituents are complex, it can be really hard to pinpoint what they have in common.
For example, let’s say you’re tasked with increasing membership in your professional association for financial advisors. If you’re going to implement an efficient, effective marketing strategy—and if you’re going to do it at scale—your first step is to develop a concrete understanding of what financial advisors generally have in common. (And it’s not enough to say “job title” or “money.”)
Once you understand what motivates a good portion of financial advisors, you can highlight the membership benefits that resonate.
In this sense, “User research” simply helps you present your ideas in a way that your users find valuable. The better you understand their needs, the better you can guide them to act the way you hope they will. User research is a deep subject, and it’s always changing because our understanding of the human psyche is always evolving. The long-term success of your organization depends on the quality of—and your commitment to—user research.
Here are some things to consider—and some helpful shortcuts—when you decide to conduct user research on your own.
1. The Five-Second Test
The five-second test is a low-cost entry into the world of user research. Simply show a piece of marketing to a user for five seconds and then ask them what they think of it.
Most commonly, you’ll show your website. Pull up the homepage (for starters), display it for only five seconds, then close it. Ask the user three questions:
- What do you remember?
- How would you describe our organization based on what you saw?
- Are you curious to learn more?
If you conduct a dozen five-second tests, you’ll immediately get a sense of how well your website is doing. If most respondents say they don’t remember much, they think your organization is confusing, and they’re not curious to learn more—well, that’s pretty telling. Or maybe they say they remember the big hero image on your website, but they don’t remember seeing your logo. Hmmmm, that’s worth noting.
2. Go Deep: Stakeholder Interviews
While surveys have great value (spoiler: they’re research method #3 in this list,) I’d argue that stakeholder interviews are a far better source of information. You can clarify participants’ responses. You can ask follow-up questions. You can ask why. Stakeholder interviews are to surveys what Netflix is to binary code. One tells the story; the other makes sure you can access the story.
But whom should you interview?
It’s tricky. On the one hand, you want to interview people who look like your target audience, but on the other hand, if you knew where to find this target audience, you’d bypass interviewing them and instead just make your sales pitch.
Our approach to recruiting stakeholders is (usually) twofold.
First, we want to interview people already connected to the organization. We’re interested in understanding why they’re engaged. We want to know what first hooked them and why they remained hooked. These might be volunteers, donors, board members, or simply advocates. (Staff should usually be interviewed only if the project is internal-facing. For public marketing and messaging research, your staff subjectivity and knowledge won’t get you far.)
Second, we want to seek out potential members or supporters who aren’t familiar with the organization. To do this, we typically draw on our professional network, or the client’s. It helps immensely if you can go into recruiting with some sense of your target audience’s demographics.
How many people should you interview?
At least 6, ideally 10.
We find that fewer than 6 interviewees produces less reliable insights—i.e., it’s hard to discover patterns. Interviewing more than 10 produces diminishing results—i.e., there’s not much you’ll discover in 12 or 15 interviews that you couldn’t discover in 10.
These numbers assume that you’ve got a good mix of interview subjects. If you’re just interviewing your staff, board members, or volunteers—instead of external, dispassionate, objective people—it doesn’t matter how many you interview, the insights will be unusable.
How do you analyze interviews?
First, record the interviews and then have them transcribed.
Second, take extensive notes. Or better yet, one person interviews while a second person takes notes.
Third, read through the interview transcription and notes. Each time you observe a potentially helpful response, place it in a spreadsheet. In the analog days, the rule of thumb was “One observation per post-it.” The spreadsheet equivalent of this rule is one concise observation per row. Make sure each entry is only one idea. If the interviewee says three or four interesting things in one answer, you’ll have three or four new entries in the spreadsheet. Over the course of a one-hour interview, you could easily end up with 50+ observations.
Fourth, in a separate column, categorize the response by what it has to do with. Responses can have multiple categorizations.
Once you’ve categorized every recorded response for every interview, sort responses by category to keep them together. And voila! You now have every meaningful thing every interviewee said about each topic.
Read through the responses. Read through them again. And again. What you’re trying to do is let patterns emerge. If you fixate on any one response, you run the risk of being misled. Remember, the opinion of one person on one topic shouldn’t influence your marketing strategy. Instead, you want to see what’s lying beneath, what the general impression is of, for example, your direct mail efforts. Reading between the lines is critical here; what do the responses infer about users’ underlying needs?
3. Go Wide: Surveys
We’ve written about surveys many times before. When well-built, surveys can give us a wide understanding of what’s working and what’s not. Remember, data kills opinions. And user surveys are all about gathering enough data to be confident that you’re heading in a smart direction. They’re giving you wide (versus deep) insight into your audiences.
But surveys are laden with potential pitfalls. It’s very easy to write surveys that unintentionally produce bad data. It’s very easy to promote surveys ineffectively and thus, get bad feedback. It’s very easy to interpret good data badly.
So before you whip up a user survey, make sure it’s airtight. Otherwise, a potentially rich source of knowledge could morph into a confusing and misleading waste of time.
Our main criteria for surveys: Don’t conduct a survey unless you’re willing to change based on what you learn. If your users, for example, tell you that—by a margin of 5 to 1—they think your brand is more “cold” than “warm” and you would prefer to be considered “warm,” you need to be prepared to change, perhaps fundamentally. Don’t put a question into a survey unless you’re willing to transform based on the responses.
Our Donor Survey Guide has tips for crafting effective surveys that can be applied beyond donors to members, volunteers, and other key audiences.
Note: You Have to Research Regularly
Most organizations conduct zero user research. And of those who do, they don’t conduct it often or deeply enough. Instead, they glance at their Google Analytics once per month, review their online reviews, and let that suffice. Make no mistake: Organizations that don’t approach user research with a genuine, open-minded sense of curiosity are missing opportunities to deliver more and deliver better.
We understand that user research seems daunting and time-consuming. It is. But it makes all of your marketing and design decisions—as well as your larger organizational direction—clearer and far more successful.
One last thing about regular research: You cannot assume you fully understand your users—and thus don’t need to do any formal research—just because your bottom line is healthy. Your bottom line may be healthy because you’re lucky, you’re the only game in town, or because you’re coasting on past successes. Remember, most companies that existed 50 years ago, even the big ones, don’t exist anymore. Why not? In part because they didn’t have their ear to the ground to recognize how markets, customers, and culture were shifting against them. (See: bookstores, Blockbuster Video, Kodak, etc.)
Some Final Thoughts on Homegrown User Research
There’s a gap between what you know and what your target audiences know. It’s a wide gap, a cavernous gap, a … gaping gap. The only way to bridge the gap is through a deep, cultural, organizational commitment to learning. User research is learning formalized.
What we’ve said here is just the tip of researchberg. There are countless other tactics available—each one with its own opportunities and pitfalls.
You probably shouldn’t conduct most of your research on your own. For starters, your biases are impossible to scrub completely from your research efforts. (And biases are the death knell of research, of course.) But it’s not just objective question-asking that a third-party researcher can offer; it’s objective question formulation. Whether you’re writing a survey, an interview script, a focus group plan, a usability script, etc., you need to ask the best possible questions in the best possible way.
We believe any organization can, and should, conduct user research on an ongoing basis. We believe that the more research you do, and the more effectively you execute it, the greater your organization’s chances of making powerful, radical, effective changes.
Good luck out there, researchers. Feel free to pick our brains about user research any time.
