Working in an enterprise environment can seem daunting at times. As an online specialist, analyzing vast amounts of quantitative data, that often comes with an enterprise organization territory, is bliss. Qualitative data on the other hand often results in sleepless nights, loss of motivation, the sporadic appearance of grey hairs and for those less fortunate, receding hairlines.

What’s next?

So what to do? In the past, I have taken it upon myself to venture out and break bend the rules a bit in order to find out more about our online visitors. The buzz word for this is *guerrilla usability testing* GUT for short… (it seems that Russ Unger/Todd Zaki Warfel are writing a book on this topic as we speak).

Along with some guts and determination, there are only a few things that you’ll really need to get started on any GUT. Essential for any research in this manner, in my case, have been Content Management friends, some free or small paid accounts** with some very reputable SaaS usability tool vendors and a plan.

Recruitment

For this specific blog post, focusing on the recruitment of an online panel via your website, you’ll need KISSinsights ($19 per month), Wufoo (free up to 100 respondents per month), MailChimp (free account is good enough for many) and, for this example, a task oriented usability testing tool.

For this last bit I used Usabilla, but there are some alternatives out there such as IntuitionHQ which offer similar functionality with different plans and pricing.

KISSinsights

Some of you might know KISSinsights. KISSinsights is a very light weight survey tool that can be placed on a website and setup in such a way that it will popup just about anywhere. You can set up KISSinsights to show in accordance with one or many different rules that you can declare on the survey’s configuration page.

By accident, I discovered that when configuring a survey, I could remove all the answers except for one. Then, while scrolling down to the “Thank You Message” and “What should the survey look like” options, I found an amazing combination:

1. “Redirect the user to another page”

2. “Send survey on response click (don’t show send button)”

What would happen if we put these together? You’re right, a single answer survey, that when answered, would lead the participant directly to a custom URL.

What one answer question could we ask? How about the following: * We are working on improving our website and services and your feedback plays a key role in our research. Would you mind helping us out in a 5 minute survey? If not, please ignore this message. Below that, KISSinsights would only show a single radio button answer with the text ‘Yes (test will start immediately)’

There you have your custom panel invite tool setup on any page that you can put the KISSinsights tag on (see your CMS friends…) which can send participants to any URL you wish. Not bad for $19.

The Custom URL

In my case, I sent my visitors to a Usabilla test. I kept the test simple and short, but interesting enough to give the participants the feeling that they were already starting to help us. The test focused simply on obtaining their opinion on the design of the homepage.

• What draws your attention the most?
• What do you find the most important on the page?
• What would you remove from the page?
• …

The test delivers on the promise that we really want our visitors’, the individual, feedback because they really are important to us.

Panel recruitment, it starts here

The reason I chose Usabilla for this test is that it allows you to use a Wufoo form at the end of the test by placing the iFrame code within the customizable Thank You Page area.

Why did I do this? Well, up to this point, we still don’t really know who the participant is. My hypothesis is, is that any one who is willing to help you once, will most likely help you again. Wufoo allows me to approach the participant and ask him/her a simple question… “Would you like to help us out again in the future?”

Now, the questionnaire was a little more thought out than that. Participants who found once enough, were sent directly to a page where I thanked them as best as anyone could online, but the others were served two extra pages.

The first of the two pages was used to collect the personal information (and subsequent an opt-in request). The second page was used to gather a minimal of segmentation data, ie. gender, age and a market specific question for the website that was used to recruit for. I will tell you more ‘why’ later on.

The participant was then sent to the same page as those who decided not to sign up, and were subsequently thanked for their time and effort.

When monkeys fly

I am always looking for an excuse to use MailChimp. Their humorous approach to UI and email marketing has probably captured the attention of many… I am one of them.

Like all the tools used so far, the key to making it all worthwhile is by connecting them. Not integration necessarily, but just being able to link them together into a single flow.

KISSinsights does not integrate with Usabilla, which in turn doesn’t integrate with Wufoo without some manual work, but Wufoo on the other hand does integrate with MailChimp.

When integrating the two, you can forward any answer from Wufoo to MailChimp, thereby allowing you to create ‘groups’ within MailChimp to which you can send subsequent invites to future tests.

Now, since not everyone who was shown the Wufoo form signed up, you are prone to receiving some error messages since, as you submit the Wufoo form, no email address is sent for participants who do not wished to participate in the future. This is one hurdle I am willing to take. I setup a filter in Gmail for these email notifications and I was done.

The juicy bits… results!

• KISSinsights 343 clickthroughs > 55% continued
• Usabilla 190 participants > 92% continued
• Wufoo 175 form starts > 67% finished
• Mailchimp 118 panel members

Overal conversion rate (clickthroughs to recruits) 34%

For those interested, the KISSinsights popup was displayed 55.000 times over a period of 5 days. So for those wanting to point it out… yes, only 0,2% of all visitors who viewed the invitation joined the panel, but still, it is a result, a result we can continue to build on and work with.

Time invested: 4 hours (3 setup, 1 analysis/reporting)
Tools used: Kampyle, Usabilla, Wufoo, Mailchimp
Conversion rate: 19%
Real value: priceless (yes, cliché I know)

DISCLAIMER exact figures and test questions have been omitted

A short intro

So there I was. A Friday afternoon just like any other, when all of a sudden my manager comes towards me and asks me if I have or can collect any data about our users opinion for an upcoming functionality.

[cue violins, choir singing and stage lighting]

This was my moment. As online analyst ninja by day and optimization guerilla by night, I clearly identified my opportunity to shine… and took it!

What do I do… and how? First let’s start with ‘why’? Internally, without going into specific details of course, a decision needed to be made on adding a new functionality to the website. The functionality had been requested by visitors and had a proven track record in positively effecting conversion as well as a user’s decision making process. We needed to know fast and effectively which design variation would suit our visitors the most and why.

Let’s get started

I have known Paul Veugen, founder of Usabilla a micro usability testing tool, for some time now and knew immediately I had to use his tool to get the data I needed. For Usabilla to be succesful, I would need users, panelmembers to participate in the test. Keep in mind that we needed the results by the following Monday! I look at the clock… it’s 2pm, where do I get a panel?

First things first… I sorted out what I needed to find out, questions and tasks that needed to be completed to prove the case and opened Usabilla in my browser. I set up the test in Usabilla, where a free account will allow you to test 1 page/design by 50 participants. I uploaded the designs we wanted to test, added some tasks, checked the test with some colleagues and saved it.

Example of a Usabilla heatmap

My next task was to open my account in Wufoo, again where a free account will get you 3 surveys and 3 reports, and set up a survey with 6 questions pertaining to the test at hand. How do I get users from Usabilla to Wufoo though? Here’s the cool bit: Usabilla allows you to embed Wufoo’s iFrame code. This will allow you to conduct an exit survey for you Usabilla participants giving you some extra information/data to work with.

Example of Wufoo survey results

Once the survey was setup and embedded in Usabilla, I again saved my test in Usabilla and launched it. I copied the test’s URL and saved it… for later. First I needed to find a panel to whom I could send the link.

Moving on

For about a month, our company has been using Kampyle (paid account, free account available too) to collect visitor feedback from our main site. The tool has proven to be very useful in collecting our visitor’s thoughts and opinions. When giving feedback, visitors are given the option to fill in their name and email address. Kampyle states that 57% of feedback users leave full contact details for you to use… I can only confirm this. So why collect contact details? This can be done for many different purposes, the main one being to conduct follow-up communication with a specific visitor. As you can probably guess, I decided to use it for a different purpose, that of creating a panel for my guerilla usability test.

Kampyle response selection

In Kampyle I created a filter to show only those feedback items in which the user left an email address. You can fine tune this filter to also include those users who also left a description (free text) and/or initiated the feedback form themselves or were presented with the form using Kampyle’s push mechanism.

Kampyle filters

I then exported the list and then imported it into Mailchimp using a free account. Mailchimp will allow you to add 500 email addresses to a list and send out 3000 emails. In other words, enough juice for 6 mailings/tests.

I quickly ran through the setup steps in Mailchimp in which I created a mailing list from the imported addresses, created a simple email template with company logo and in company colors, wrote the invitation and created a call-to-action button with the link to the Usabilla test. I had a colleague check the contents to make sure no typo’s and incorrect grammar was used. Only one thing left to do… send out the invites!

Test Result Funnel

100% emails sent
48% opened email
33% clicked to start test (click through rate)
21% started test
19% completed test (soft conversion)

Mailchimp result messages are very motivating

Looking into my crystal ball

As I mentioned before, all of this was set up in 3 hours time, from start to finish. What really surprised me was the high response rate. 48% opened the email and 33% clicked through to start the test. These are great results. Why did this happen? Here are my thoughts and conclusions on the matter:

• Visitors who previously left feedback on our site (positive or negative) were inclined to share their thoughts and opinions with us again if asked.
• Including phrases in the invitation such as ‘we appreciate your feedback so far’, ‘your feedback is helping us improve our site’ and ‘results from this test will be used to make decisions on adding/improving functionalities on the site that will made it easier for us to accomplish our goal, proved to be real winners.
• Including a clear call-to-action button and estimated completion time contributed to convincing receivers to participate.

Example call-to-action used showing that this link was click by 97.2% of recipients

We even received compliments from participants who stated that they were glad that we are taking their feedback so seriously!

We were very happy with the 19% conversion rate, but even happier to find out that we are in a position to quickly setup and send out invitations to micro usability tests. We found the answers we need with 19% of the participants without giving any incentive other than a guarantee that their voices would be heard and acted on. In the future we will perform similar tests and we are confident that we will get the same results as far as participation is concerned.