In a past blog post I described in some detail some lean and mean methods to perform remote usability tests. The article is steadily bringing visitors to my blog, visitors, who like myself, need to find uncommon methods for performing online research in order to convince the corporate-powers-that-be, that what we can do is useful and critical to improving any company’s online services.
Working in an enterprise environment can seem daunting at times. As an online specialist, analyzing vast amounts of quantitative data, that often comes with an enterprise organization territory, is bliss. Qualitative data on the other hand often results in sleepless nights, loss of motivation, the sporadic appearance of grey hairs and for those less fortunate, receding hairlines.
So what to do? In the past, I have taken it upon myself to venture out and
break bend the rules a bit in order to find out more about our online visitors. The buzz word for this is *guerrilla usability testing* GUT for short… (it seems that Russ Unger/Todd Zaki Warfel are writing a book on this topic as we speak).
Along with some guts and determination, there are only a few things that you’ll really need to get started on any GUT. Essential for any research in this manner, in my case, have been Content Management friends, some free or small paid accounts** with some very reputable SaaS usability tool vendors and a plan.** I often paid for tools myself. The insights gained from the tests that I did often resulted in either reimbursement, a raise or a promotion… so go on, take the risk, show them what you are made of. If they like it, they’ll let you know, otherwise you are working for the wrong kind of company.
For this specific blog post, focusing on the recruitment of an online panel via your website, you’ll need KISSinsights/Qualaroo ($19 per month), Wufoo (free up to 100 respondents per month), MailChimp (free account is good enough for many) and, for this example, a task oriented usability testing tool.
[alert color=yellow title=”KISSinsights now Qualaroo” align=center]As of 31 July 2012, KISSinsights will continue as Qualaroo. Read More[/alert]
Some of you might know KISSinsights. KISSinsights is a very light weight survey tool that can be placed on a website and setup in such a way that it will popup just about anywhere. You can set up KISSinsights to show in accordance with one or many different rules that you can declare on the survey’s configuration page.
By accident, I discovered that when configuring a survey, I could remove all the answers except for one. Then, while scrolling down to the “Thank You Message” and “What should the survey look like” options, I found an amazing combination:
1. “Redirect the user to another page”
2. “Send survey on response click (don’t show send button)”
What would happen if we put these together? You’re right, a single answer survey, that when answered, would lead the participant directly to a custom URL.
What one answer question could we ask? How about the following: * We are working on improving our website and services and your feedback plays a key role in our research. Would you mind helping us out in a 5 minute survey? If not, please ignore this message. Below that, KISSinsights would only show a single radio button answer with the text ‘Yes (test will start immediately)’
There you have your custom panel invite tool setup on any page that you can put the KISSinsights tag on (see your CMS friends…) which can send participants to any URL you wish. Not bad for $19.
The Custom URL
In my case, I sent my visitors to a Usabilla test. I kept the test simple and short, but interesting enough to give the participants the feeling that they were already starting to help us. The test focused simply on obtaining their opinion on the design of the homepage.
- What draws your attention the most?
- What do you find the most important on the page?
- What would you remove from the page?
The test delivers on the promise that we really want our visitors’, the individual, feedback because they really are important to us.
Panel recruitment, it starts here
The reason I chose Usabilla for this test is that it allows you to use a Wufoo form at the end of the test by placing the iFrame code within the customizable Thank You Page area.
Why did I do this? Well, up to this point, we still don’t really know who the participant is. My hypothesis is, is that any one who is willing to help you once, will most likely help you again. Wufoo allows me to approach the participant and ask him/her a simple question… “Would you like to help us out again in the future?”
Now, the questionnaire was a little more thought out than that. Participants who found once enough, were sent directly to a page where I thanked them as best as anyone could online, but the others were served two extra pages.
The first of the two pages was used to collect the personal information (and subsequent an opt-in request). The second page was used to gather a minimal of segmentation data, ie. gender, age and a market specific question for the website that was used to recruit for. I will tell you more ‘why’ later on.
The participant was then sent to the same page as those who decided not to sign up, and were subsequently thanked for their time and effort.
When monkeys fly
I am always looking for an excuse to use MailChimp. Their humorous approach to UI and email marketing has probably captured the attention of many… I am one of them.
Like all the tools used so far, the key to making it all worthwhile is by connecting them. Not integration necessarily, but just being able to link them together into a single flow.
KISSinsights does not integrate with Usabilla, which in turn doesn’t integrate with Wufoo without some manual work, but Wufoo on the other hand does integrate with MailChimp.
When integrating the two, you can forward any answer from Wufoo to MailChimp, thereby allowing you to create ‘groups’ within MailChimp to which you can send subsequent invites to future tests.
Now, since not everyone who was shown the Wufoo form signed up, you are prone to receiving some error messages since, as you submit the Wufoo form, no email address is sent for participants who do not wished to participate in the future. This is one hurdle I am willing to take. I setup a filter in Gmail for these email notifications and I was done.
The juicy bits… results!
- KISSinsights 343 clickthroughs > 55% continued
- Usabilla 190 participants > 92% continued
- Wufoo 175 form starts > 67% finished
- Mailchimp 118 panel members
Overal conversion rate (clickthroughs to recruits) 34%
For those interested, the KISSinsights popup was displayed 55.000 times over a period of 5 days. So for those wanting to point it out… yes, only 0,2% of all visitors who viewed the invitation joined the panel, but still, it is a result, a result we can continue to build on and work with.