Free usability testing, no really!

First the facts…

Time invested: 4 hours (3 setup, 1 analysis/reporting)
Tools used: Kampyle, Usabilla, Wufoo, Mailchimp
Conversion rate: 19%
Real value: priceless (yes, cliché I know)

DISCLAIMER exact figures and test questions have been omitted

A short intro

So there I was. A Friday afternoon just like any other, when all of a sudden my manager comes towards me and asks me if I have or can collect any data about our users opinion for an upcoming functionality.

[cue violins, choir singing and stage lighting]

This was my moment. As online analyst ninja by day and optimization guerilla by night, I clearly identified my opportunity to shine… and took it!

What do I do… and how? First let’s start with ‘why’? Internally, without going into specific details of course, a decision needed to be made on adding a new functionality to the website. The functionality had been requested by visitors and had a proven track record in positively effecting conversion as well as a user’s decision making process. We needed to know fast and effectively which design variation would suit our visitors the most and why.

Let’s get started

I have known Paul Veugen, founder of Usabilla a micro usability testing tool, for some time now and knew immediately I had to use his tool to get the data I needed. For Usabilla to be succesful, I would need users, panelmembers to participate in the test. Keep in mind that we needed the results by the following Monday! I look at the clock… it’s 2pm, where do I get a panel?

First things first… I sorted out what I needed to find out, questions and tasks that needed to be completed to prove the case and opened Usabilla in my browser. I set up the test in Usabilla, where a free account will allow you to test 1 page/design by 50 participants. I uploaded the designs we wanted to test, added some tasks, checked the test with some colleagues and saved it.

My next task was to open my account in Wufoo, again where a free account will get you 3 surveys and 3 reports, and set up a survey with 6 questions pertaining to the test at hand. How do I get users from Usabilla to Wufoo though? Here’s the cool bit: Usabilla allows you to embed Wufoo’s iFrame code. This will allow you to conduct an exit survey for you Usabilla participants giving you some extra information/data to work with.

Once the survey was setup and embedded in Usabilla, I again saved my test in Usabilla and launched it. I copied the test’s URL and saved it… for later. First I needed to find a panel to whom I could send the link.

NOTE Social Media channels were used too, but yielded lower number of quick responding participants.

Moving on

For about a month, our company has been using Kampyle (paid account, free account available too) to collect visitor feedback from our main site. The tool has proven to be very useful in collecting our visitor’s thoughts and opinions. When giving feedback, visitors are given the option to fill in their name and email address. Kampyle states that 57% of feedback users leave full contact details for you to use… I can only confirm this. So why collect contact details? This can be done for many different purposes, the main one being to conduct follow-up communication with a specific visitor. As you can probably guess, I decided to use it for a different purpose, that of creating a panel for my guerilla usability test.

In Kampyle I created a filter to show only those feedback items in which the user left an email address. You can fine tune this filter to also include those users who also left a description (free text) and/or initiated the feedback form themselves or were presented with the form using Kampyle’s push mechanism.

I then exported the list and then imported it into Mailchimp using a free account. Mailchimp will allow you to add 500 email addresses to a list and send out 3000 emails. In other words, enough juice for 6 mailings/tests.

I quickly ran through the setup steps in Mailchimp in which I created a mailing list from the imported addresses, created a simple email template with company logo and in company colors, wrote the invitation and created a call-to-action button with the link to the Usabilla test. I had a colleague check the contents to make sure no typo’s and incorrect grammar was used. Only one thing left to do… send out the invites!

Test Result Funnel

100% emails sent
48% opened email
33% clicked to start test (click through rate)
21% started test
19% completed test (soft conversion)

Looking into my crystal ball

As I mentioned before, all of this was set up in 3 hours time, from start to finish. What really surprised me was the high response rate. 48% opened the email and 33% clicked through to start the test. These are great results. Why did this happen? Here are my thoughts and conclusions on the matter:

• Visitors who previously left feedback on our site (positive or negative) were inclined to share their thoughts and opinions with us again if asked.
• Including phrases in the invitation such as ‘we appreciate your feedback so far’, ‘your feedback is helping us improve our site’ and ‘results from this test will be used to make decisions on adding/improving functionalities on the site that will made it easier for us to accomplish our goal, proved to be real winners.
• Including a clear call-to-action button and estimated completion time contributed to convincing receivers to participate.

We even received compliments from participants who stated that they were glad that we are taking their feedback so seriously!

We were very happy with the 19% conversion rate, but even happier to find out that we are in a position to quickly setup and send out invitations to micro usability tests. We found the answers we need with 19% of the participants without giving any incentive other than a guarantee that their voices would be heard and acted on. In the future we will perform similar tests and we are confident that we will get the same results as far as participation is concerned.