An open letter: ‘How User Feedback Tools Fail’

An open letter: ‘How User Feedback Tools Fail’

Share these actual insights
Creating user feedback tools is no longer the challenge, giving value is and many fail at this. Let me try to tell you why I think this is.

I was not sure if I should write down my thoughts on this topic or not… let alone publish them online.

Ok, I guess I made up my mind on that conundrum. Let me point out though that this is not an attack on anyone, any company, or any tool. It’s a mere observation.

Let’s break it down first

While working on my online course about user feedback tools I discovered a common error that many user feedback tool vendors are making.

You see, there is a relatively easy process when it comes to user feedback research and putting it to use for online optimisation.

  1. Collect Feedback
  2. Analyse Feedback
  3. Report Feedback (and hopefully you take action on the insights)

‘Feedback Collection’ Galore!

All the tools out there on the market today do a great job at collecting feedback. All vendors offer passive collection methods, the more premium vendors allow you to actively approach users, or even target them based on behavior.

I applaud these techniques, I really do because I have been able to solve some real issues using user submitted feedback and have even considered replacing regular NPS surveys with more page level feedback since it is more contextual, and more effective for solving user experience issues.

Analysis Fail One

I have accounts at almost all the vendors out there, and I have used almost all tools out there. I can safely say that with only a few exceptions (few as in 1 or 2) the analysis capabilities in user feedback tools are lacking the necessary punch.

User Feedback Tools Price vs Analytics Features

In the chart above, I tried to plot the tools I reviewed on two axis.

  1. Pricing (X-axis)
  2. Analysis functionalities (Y-axis)

I defined Level 1 as being able to view quantitative data about the number of feedback items received, number of feedback items per pre-defined category (if available), and the average scores given.

In Level 1 you would also have access to the qualitative data, in this case the answers to the open-ended questions.

For Level 2 I added analysis functionalities such as filters for text, categories, scores, pages, date ranges, and more than 1 export options.

Level 3 would also, just to name few, contain feature like dynamic word clouds, and the ability to cross reference quantitative answers (radio button answers etc) with open-ended question answers.

Clustered based on functionality

As you can see many tools are clustered simply based on the analysis functionalities they offer, or should I say don’t offer.

This is where the big opportunities lie. Gone are the barriers to collect, gone are the barriers to target users, and gone are the barriers to collect on mobile devices.

It is time to separate the men from the boys.

While vendors continue to focus on collection methods, analysis functionalities are failing. A decision that, from personal experience and talks with insiders, is costing vendors customers.

Churn is massive.

Opportunities for user feedback tool vendors

Vendors, in my opinion, can truly distinguish themselves simply by starting to investigate and invest in functionalities for the analysts, the marketeers, the people who need to use the tools, and eventually recommend the renewal of the tool’s subscription.

Stop focussing on collecting, we’ve crossed that bridge. Start making your tools more than eye-candy for management, make them leverage the knowledge and motivation of intrapreneurs, the foot soldiers in the online realm, the people who give your company meaning and value in return.

Value of user feedback tools does not rest solely in the collection capabilities, it never should have.

Analysis and reporting are the two ignored selling points of your power to give us insights into our user’s motivations and fears.

Harnas it please. Today.

Interested in learning more about user feedback survey tools?

Since you made it this far, I wanted to thank you for your attention. If you are interested in learning more about user feedback survey tools I invite you to join my course for free.

I would be more than interested in receiving your thoughts on my open letter above, and of course my course.

Note: This article was orignally posted on Medium on march 12th 2014 by Matthew Niederberger

About Matthew Niederberger

Matthew works as Conversion Optimisation Manager at Ziggo BV. In his free time he enjoys family life as well as digging into online user research material whilst frequently generating some of his own, which he freely shares here on


  1. Thanks for an insightful article.

    My gripe with the vendors at the moment is the amount they charge.

    To collect the basic usable minimum of user feedback, you need certain functionalities from these tools.
    However at the moment you can’t get that with the free of basic plans these vendors offer.

    At the moment it feels to me like bait and switch. They bait you in with a free or low-cost basic plan and then you realise, you can’t achieve much with that plan and you are obliged to upgrade.

    Matthew, this might be an opportunity for you to commission somebody to build a tool with all the capabilities you envisage! Shake-up that market place with your inside knowledge 

  2. Hey PK.

    Thanks for your encouraging words. I have played around with the idea you mentioned, I actually have some rough sketches and flows in my notebooks. I have started to learn how to program so that I can bootstrap the project.

    As far as pricing and plans, I agree. The fees are high, but like my chart shows, I think that most of these companies in level 2 are at the far edge of pricing. They will need to up their game and offer more for the analysts, features will only take you so far. How many ways are there to collect feedback? Collecting is only 10% of the power of the tool. iPerceptions have some really great functions from which other vendors can learn.

    Shakin’ it up,
    Matthew 🙂

Leave a Reply

FREE Course!

Get FREE access to my User Feedback Fundamentals course at Udemy, a $29 value. Just sign-up to my newsletter.
Unsubscribe at anytime and keep access to the course.

Nope, we don't deal in spam, not even the canned sort.