Originally published at: http://www.sitepoint.com/choosing-usability-tests-participants/
In my last article, we talked about ways to really nail down your usability goals before you even think about testing.
Once your goals are clear, you’re ready to hone your test planning to meet those specific goals. There are many tests to choose from, and many types of people to recruit, so narrowing your focus really helps get you closer to the result you’re looking for.
Depending on your hypothesis and what you want to learn, sometimes your usability test might not even require testing the actual product.
Source: Usability Testing
Let’s take a look at the different categories of usability tests, how to find the appropriate testing audience, and how to make it digestible with a simple usability plan.
Types of Test
Deciding which style of test to administer is a pivotal decision in the entire process of usability testing, so don’t take it lightly. On the bright side, the more concrete your usability goals are, the more smoothly the selection process will go.
But no matter what type of test you choose, you should always start with a pilot test. Many people like to gloss over this, but sacrificing a little extra time for a pilot test almost always pays off.
What’s a Pilot Test?
“A test!? No-one mentioned a test to me “
Photo: The Library of Congress
Don’t worry, you won’t need make friends with aviators.
Pilot testing is a test run of your greater user test. In A Practical Guide to Usability Testing, Joseph S. Dumas and Janice C. Redish call pilot tests a "dress rehearsal for the usability test to follow."
You will conduct the test and collect the data in the same way you would a real test, but the difference is that you won’t analyze or include this data. You are, quite literally, testing your test.
This may seem like a waste of time — and you will likely be tempted to just jump head-long into the actual tests — but pilot tests are highly recommended. This is because in most cases, something WILL go wrong with your first test. Whether it is technical problems, human error, or a situational occurrence, it’s rare that a first test session goes well – or even adequately.
However, the idea is that these tests should be as scientific and precise as possible. If you want the most reliable data, run a pilot test or two until you feel you understand all the variables thoroughly, and have ironed out all the kinks.
The Types of Tests
I’ve written a Usability Testing Guide, that delves more deeply into the specifics of each type of user testing method if you're interested. But for this article, we’ll give you an overview so you know what the landscape looks like.
Source: Nielsen Norman Group
Christian Rohrer, McAfee’s Chief Design Officer, explains in an article for the Nielsen Norman Group the distinctions between the types of tests. While Christian uses a complex three-dimensional framework to explain the their differences, for simplicity’s sake we’re going to focus on his division based on how the product is used.
- Scripted use of the product : These tests focus on specific usage aspects. The degree of scripting varies, with more scripting generating more controlled data.
- Decontextualized use of the product : Tests that don’t use the product — at least in the actual testing phase — are designed for broader understanding of topics like UX or generating ideas.
- Natural (and near-natural) use of the product : These tests seek to understand common usage behaviors and trends with the product, prioritizing real user data at the cost of control. This tests fundamental assumptions of how a product is being used.
- Hybrid : Hybrid tests are creative and non-traditional tests. Geared towards understanding the users’ mentality, these tests vary in what they can accomplish.