"Your website sucks!" - Why users actions don't lie and how to deal with it
Have you ever experienced a situation when you are working with someone and something surely has been made wrong but your coworker tells you it’s definitely not and you have no chance to proof him or her wrong? Well, I have a solution for you in case of pitfalls in design. I’m talking about user testing.
User Testing is an eye-opening method and I love to spread the word about it, because so many companies still don’t know what it is and that really is a pity. However, especially startups or small companies are doing something in that direction, but often it is not as effective as user testing and it can be made way more effective. So let me first describe what it is. Your website or app is made for a purpose, right? There are actions that users should be able to achieve using your website. When user testing, these actions are formulated into tasks, you take some people out of your target audience and sit them in front of your website or app and let them do the tasks while speaking out loud what they do and think. That specific method is called Think Aloud. Sounds simple right? Well, you would wonder what mistakes were uncovered with that method and beyond that how convincing that method is. Taken the situation above where you have that colleague that just won’t agree with you, reimagined in a user testing setting. Of course that person could tell that the user is too stupid, but if that user is right out of your target group, that’s rarely the case. So, your coworker would have to accept that there is an issue that needs to be fixed.
That’s not the only benefit of user testing. Let’s first talk about the pros before going into some downsides as well.
One big pro is that user testing is an inexpensive method as you only need 5 users out of your target audience to do a round of user testing. Why 5? Well, there is a study that shows that 5 is the number with which you can uncover 85% of usability issues while it takes much more to reach a percentage above that. It was also uncovered that it is more effective to do 3 rounds of user testing with 5 participants each round than one round with 15 participants (Nielsen & Landauer, 1993). So besides the time you have to invest in finding participants and maybe offer them a small incentive, it’s is really no big investment. One short note on family and friends as participants: As long as your friends and family don’t interact too often with your platform they could also be proper participants. However, they should clearly refer to your target audience, because it makes no sense to test a system made for IT experts with your dad who barely knows how to write an email.
Another good thing about user testing is that it is very flexible. You can almost test everything, from early prototypes of a vague idea to new features for an established system. Each way, you will get amazing insights in a quick way which can prevent you from going into wrong directions in early stages, so that you are able to work “lean”.
Furthermore, the method is easy to learn and apply. In my Guideline “How to: Usability & UX” I explain how to approach user testing step by step (using a simple framework) and I offer plain forms for a script that you can follow during the test and a consent and recording form for the participant to sign along with some examples for tasks and scenarios as well as a severity ranking for the analysis. If you have a bit more budget you can also use services like usertesting.de and alike to get your website or app tested.
As promised I will also go into some downsides to user testing.
First of all, of course, we have an unnatural situation, because normally people do not think out loud while sitting in front of a computer. This can be awkward in the beginning, but you should simply remind and encourage your participants to do so.
Second, people tend to filter their statements as they don’t want to hurt other people’s feelings, so you have to clearly say that your feelings won’t be hurt by negative comments and that you are rather looking to find mistakes and problems. Tell your participants they can tell their honest opinions and reactions.
Lastly, as much as I love user testing, it is no panacea itself. You still have to do your research in advance to get the right participants to evaluate your system with the right people. Further, you have to interpret and prioritize the outcomes so that you can improve your system and do another user testing as it is an iterative process.
In the end, it is not done by conducting a user test. You have to evaluate, analyze and interpret the outcomes correctly. Here, it is important to list all issues found, how often they occurred during the test and how often users come along that issues as well as the severity of that issue and the ease of fixing. After you’ve done that you can implement the changes and do another round of user testing to see whether the issues were solved or whether the same problem is still present. In the latter case, you have to come up with another solution. Don’t forget that in the best case you would do a second round of user testing with different participants than in the first round as they will potentially uncover new issues.
All in all, user testing really is a great tool to uncover usability issues and improve the user experience of your system. As long as you approach it with a clear structure defining your goals, exploring the questions you want to have answered and consider practical issues, not much can go wrong. Of course, then it is also important to evaluate, analyze and interpret the issued found correctly.
What about you? Did you already do a user test or something similar and what were your findings?