Not All Feedback Is Created Equal
Product feedback comes from various sources and it's important to weigh the credibility and priority of that feedback.
If I were to show my sister virtually any product I’ve worked on in my career and ask her for advice. I’d get something along the lines of “what is this, why doesn’t it do X?” We’ve all received that type of feedback, and it’s a clear indicator that you either are building the wrong thing or (more likely) you are asking for feedback from the wrong audience. However, feedback runs a spectrum, and it won’t always be so easy to spot feedback worth paying attention to and feedback worth ignoring.
When working at a growing organization, collaboration across company functions will invariably results in a sense of product ownership across many teams. That’s a great thing. The more your marketing team and business development team and operations team cares about the product, the better aligned you’ll be as an organization. That said, you’ll also likely face competing feedback. This is natural. Each team has its own agenda and needs. Those needs may not map directly back to the customer.
Combine internal feedback with customer feedback, and you will quickly find yourself in need of a filtering system. Don’t worry, we don’t need to reach for ChatGPT for this system. We just need mental models to help us categorize and weigh feedback appropriately. When you’ve developed this mental model, the filtering will happen naturally and almost automatically. Each person’s model will be different, but we can build a framework around how to develop the model. Before we do, let’s summarize the various locations from which you might receive feedback.
Customers
CEO
Product team members
Sales/BD
Marketing
Operations
Design
Support
Enterprise customers
Free plan customers
Friends
Family
Paying, non-enterprise plan customers
Hacker News (you know it happens)
Reddit
Twitter
…
That’s a lot of sources. And the list doesn’t even include all the possibilities. You’ll get feedback from random Facebook groups and some Medium article will mention your product and some of its shortcomings in the eyes of the author. If your company has social listening tools in place, you can expect the volume of feedback to be immense.
But don’t worry, the filtering model you’ll develop will serve you well. Let’s talk about how to develop a framework that will allow you to deploy your own model.
Step One: Understand the Business
Feedback must be filtered through a process that works backwards. The first point in which to work backwards from is the business itself. Do you know the business? How does the business make money? What’s your company’s north star?
These questions seem simple, but it is so easy in product development to chase after things that are interesting and innovative but don’t serve the business. The shiny thing syndrome can result in product and company failure if not kept in check. As I mentioned in my last post, experimentation is definitely part of the product development process, but even then, experimentation must be built around the goals of the business.
Step Two: Understand the Source
The source of the feedback matters. You may want to weigh the feedback of a famous person who is using your product and tweeting about it more than you might weigh the feedback of some other user, but is this famous person’s feedback actually more important? It depends.
Here are some things to consider when trying to understand the value of the source:
How often do they use the product?
Are they paying for the product?
Is the product core to something they do daily or weekly?
If the product disappeared for the person, how upset would they be?
How close are they to the product?
Does the source stand to gain in some way unrelated to the product if the feedback is implemented?
The last point is an interesting one but definitely worth considering. Feedback for the sake of personal or other gain that does not directly make the product better may be hard to decipher, but it’s important to try.
In a famous example of product feedback that was ultimately implemented, a tech consultant named Chris Messina suggested the concept of hashtags for Twitter via a public tweet. Twitter’s product team had to consider the source as they evaluated this feedback. Twitter was only a year old, Chris was an active user, and his feedback seemed to resonate with others. Chris satisfied many of the requirements as a good source of feedback, but Twitter was still hesitant to implement the idea. Chris did stand to gain from his feedback being implemented in the sense that it would be his claim to fame. But it was genuinely useful, and Twitter, as we know, ultimately implemented the feature—a feature that has since been copied in almost every other social app on the market.
The source of the feedback is important in helping you filter the feedback into meaningful product development or filtering it out entirely.
Step Three: Consider the Impact
Along with varied sources of feedback, you’ll also get varied types of feedback. You’ll see requests and suggestions ranging from typeface changes to color improvements to entirely new features. While you have your own roadmap (or Loosely Held Ideas), that roadmap should be shaped by the needs of your customers and the needs of the business.
To help decide what’s important, you should consider the impact. There are many frameworks out there that try to help with this, but I tend to think most of them over-complicate the equation. For example, the RICE framework (risk, impact, confidence, and effort), includes “impact” as part of the analysis. I believe impact is the output of effort and reach. Risk is inherent in every product or feature you ship. Confidence is a guess. It’s relatively simple to understand the reach of a particular feature improvement or product release. It’s hard to understand the effort because estimating engineering effort is a difficult problem. But if you can get close on both of those things, you will have a rough approximation of the impact of the work you are considering.
Rather than considering impact as an input into a decision to work on something or not, consider impact the output.
The fun and beauty of working in product is that you will never have a shortage of feedback and suggestions. The maddening part of working in product is that you will never have a shortage of feedback and suggestions.
It’s important not to look inward and simply trust your gut. Instead, you should involve other. You should encourage feedback. You should look at everything with an open mind. But then run it all through your filtering model and separate the good feedback from the bad.