Free usability testing, no really!

Share these actual insights

First the facts…

Time invested: 4 hours (3 setup, 1 analysis/reporting)
Tools used: Kampyle, Usabilla, Wufoo, Mailchimp
Conversion rate: 19%
Real value: priceless (yes, cliché I know)

DISCLAIMER exact figures and test questions have been omitted

A short intro

So there I was. A Friday afternoon just like any other, when all of a sudden my manager comes towards me and asks me if I have or can collect any data about our users opinion for an upcoming functionality.

[cue violins, choir singing and stage lighting]

This was my moment. As online analyst ninja by day and optimization guerilla by night, I clearly identified my opportunity to shine… and took it!

What do I do… and how? First let’s start with ‘why’? Internally, without going into specific details of course, a decision needed to be made on adding a new functionality to the website. The functionality had been requested by visitors and had a proven track record in positively effecting conversion as well as a user’s decision making process. We needed to know fast and effectively which design variation would suit our visitors the most and why.

Let’s get started

I have known Paul Veugen, founder of Usabilla a micro usability testing tool, for some time now and knew immediately I had to use his tool to get the data I needed. For Usabilla to be succesful, I would need users, panelmembers to participate in the test. Keep in mind that we needed the results by the following Monday! I look at the clock… it’s 2pm, where do I get a panel?

First things first… I sorted out what I needed to find out, questions and tasks that needed to be completed to prove the case and opened Usabilla in my browser. I set up the test in Usabilla, where a free account will allow you to test 1 page/design by 50 participants. I uploaded the designs we wanted to test, added some tasks, checked the test with some colleagues and saved it.

Example of a Usabilla heatmap

Example of a Usabilla heatmap

My next task was to open my account in Wufoo, again where a free account will get you 3 surveys and 3 reports, and set up a survey with 6 questions pertaining to the test at hand. How do I get users from Usabilla to Wufoo though? Here’s the cool bit: Usabilla allows you to embed Wufoo’s iFrame code. This will allow you to conduct an exit survey for you Usabilla participants giving you some extra information/data to work with.

Example of Wufoo survey results

Example of Wufoo survey results

Once the survey was setup and embedded in Usabilla, I again saved my test in Usabilla and launched it. I copied the test’s URL and saved it… for later. First I needed to find a panel to whom I could send the link.

NOTE Social Media channels were used too, but yielded lower number of quick responding participants.

Moving on

For about a month, our company has been using Kampyle (paid account, free account available too) to collect visitor feedback from our main site. The tool has proven to be very useful in collecting our visitor’s thoughts and opinions. When giving feedback, visitors are given the option to fill in their name and email address. Kampyle states that 57% of feedback users leave full contact details for you to use… I can only confirm this. So why collect contact details? This can be done for many different purposes, the main one being to conduct follow-up communication with a specific visitor. As you can probably guess, I decided to use it for a different purpose, that of creating a panel for my guerilla usability test.

Kampyle selection

Kampyle response selection

In Kampyle I created a filter to show only those feedback items in which the user left an email address. You can fine tune this filter to also include those users who also left a description (free text) and/or initiated the feedback form themselves or were presented with the form using Kampyle’s push mechanism.

Kampyle filters

Kampyle filters

I then exported the list and then imported it into Mailchimp using a free account. Mailchimp will allow you to add 500 email addresses to a list and send out 3000 emails. In other words, enough juice for 6 mailings/tests.

I quickly ran through the setup steps in Mailchimp in which I created a mailing list from the imported addresses, created a simple email template with company logo and in company colors, wrote the invitation and created a call-to-action button with the link to the Usabilla test. I had a colleague check the contents to make sure no typo’s and incorrect grammar was used. Only one thing left to do… send out the invites!

Test Result Funnel

100% emails sent
48% opened email
33% clicked to start test (click through rate)
21% started test
19% completed test (soft conversion)

Mailchimp result messages

Mailchimp result messages are very motivating

Looking into my crystal ball

As I mentioned before, all of this was set up in 3 hours time, from start to finish. What really surprised me was the high response rate. 48% opened the email and 33% clicked through to start the test. These are great results. Why did this happen? Here are my thoughts and conclusions on the matter:

    • Visitors who previously left feedback on our site (positive or negative) were inclined to share their thoughts and opinions with us again if asked.
    • Including phrases in the invitation such as ‘we appreciate your feedback so far’, ‘your feedback is helping us improve our site’ and ‘results from this test will be used to make decisions on adding/improving functionalities on the site that will made it easier for us to accomplish our goal, proved to be real winners.
    • Including a clear call-to-action button and estimated completion time contributed to convincing receivers to participate.
Example call-to-action

Example call-to-action used showing that this link was click by 97.2% of recipients

We even received compliments from participants who stated that they were glad that we are taking their feedback so seriously!

We were very happy with the 19% conversion rate, but even happier to find out that we are in a position to quickly setup and send out invitations to micro usability tests. We found the answers we need with 19% of the participants without giving any incentive other than a guarantee that their voices would be heard and acted on. In the future we will perform similar tests and we are confident that we will get the same results as far as participation is concerned.

Receive FREE updates

Get FREE updates and UX product discounts directly in your inbox by simply signing up for my newsletter. You can unsubscribe at any time.

Your information will never be shared, period!
About Matthew Niederberger

Matthew works as Conversion Optimisation Manager at Ziggo BV. In his free time he enjoys family life as well as digging into online user research material whilst frequently generating some of his own, which he freely shares here on actualinsights.com.

Comments

  1. That was a great usability testing case study. I have been getting more into conversion rate optimization and so far I have been focusing on the heatmap and user-site interaction aspect of it. Those combined with direct user feedback such as this would provide invaluable insights.

  2. Steve says:

    Would like to mention this budget-friendly provider: http://www.usabiliTEST.com

Trackbacks

  1. [...] Matthew’s post Free usability testing, no really! for the full story. TweetShare 0savesSave Enjoyed this post? Subscribe to our RSS [...]

  2. [...] Like many other form situations, you will always get some Mickey Mouse entries with fake names, email addresses and phone number, but just take these for granted. In a previous blog post I discussed how you can use the collecting of email addresses to your advantage when setting up simple usability tests. [...]

  3. [...] several great tools to set up a guerrilla research study I took it upon myself to get some sense of the recognition of trust logos among consumers and its [...]

  4. [...] a past blog post I described in some detail some lean and mean methods to perform remote usability tests. The [...]

  5. [...] resources needed to research, document, setup and execute usability testing are not dramatic, but they aren’t effortless [...]

  6. […] several great tools to set up a guerrilla research study I took it upon myself to get some sense of the recognition of trust logos among consumers and its […]

Leave a Reply

Share these actual insights

Get over 150+ Top Udemy Courses for only $10 each!