Best Questions for Usability Testing – What to Ask

Overview

• We are sharing our best questions for usability testing
• Questions are provided with a broader context – this is not just a simple list of questions
• Three distinct phases – opening questions, central questions, and closing questions
• Listen and guide – navigate from abstract observation to concrete and tangible comments
• Following up with a more structured approach (scales, SUS, Likert’s) could be beneficial

Three phases – each with its own questions

Usability testing, user research, and user testing are standard services we provide. They are also some of the most engaging since they involve a lot of interaction with real, live, human beings. Today, we are sharing our best questions for usability testing.

During our usability tests with users, we ask them many different questions. Besides learning about their habits, expectations, and them as a person, we also learn about the product we are testing.

Based on our collective experience, I decided to share some of our favorite usability questions. This is not an exhaustive list, and your questions will likely be different. Our experience showed that some questions and approaches are common to our usability testing sessions. These questions stood the test of time over and over again. They also allowed us to understand the products and people using them. While your mileage will vary, I think that sharing these questions with you might help you get more valuable insights from users.

At the top level, we are grouping our questions in three distinct, but very connected categories. They are introductory questions, central questions, and closing questions. All of them, combined, make the list of our best questions for usability testing.

Now, hold tight, I’m going to take you through all categories and associated questions. Instead of only listing the items, I will give you some background and reasons why we ask those questions. Sharing them without the reasoning behind them seemed a bit superficial since It would not give you significant value.

Introductory best questions for usability testing

These are the questions we ask even before the testing starts. We will open with some broad questions to make the user feel comfortable. A large part of this introduction is our monologue. We will go and say things like:

“Thank you for coming to this usability testing. We appreciate your time. I’m going to show you around the product/service/website we are going to test today. I want you to know that we are not testing you and your abilities to figure out how to use the product. We are trying to see if this product is developed with people and their needs in mind. And you will be able to help us with that.”

The purpose of this introduction is to get the person to relax. To encourage them to be open and honest in their feedback since we are not testing them – we are testing the product. If they can achieve something and carry out the desired action, it’s not them who are doing it wrong – it’s the product. If we notice that the person is a bit nervous, we will start even more casually and ask them if they had trouble finding our office. We will chat about the weather. Sometimes they will comment on our office space…

Relax

Do your best to make them feel as comfortable as possible. Some people will feel relaxed and chatty by their own nature. Others might be a bit more nervous and need a bit more time to relax. Take your time – don’t rush it. This is one of the reasons we never schedule tests one right after the other without any buffer time. Having a buffer time is a good idea since some people might get stuck in the traffic or otherwise late. You don’t want to rush them and make them feel nervous even more.

After this initial welcome and relaxing conversations, we turn to more topical questions.

If I tell you that the Company ABC has developed a tool XYZ which helps/makes/does (something) – what would be your expectations from that product?

Here, you are trying to establish a frame of reference. You want to see and understand what would be the initial expectations form a product you are testing. For example, if you say that the Company has developed a website where people can rent a boat and sail along the Croatian coast, you want to know what that person would expect from that kind of service. We do this even before we show the first screen or give them any detailed information about the product we are testing.

Did you ever use a product similar to the one I mentioned? What did you (dis)like about it? Can you share some of your views?

Again, we want to establish an understanding of the user’s expectations and their familiarity with similar products or services.

Central questions

This is the section where most of your questions will depend on the actual testing scenarios and the product you are testing. But we still do have some good ideas on how to make most of the user you are testing. Also, this is the part where questions asked by a user could be more insightful than your own questions.

It is critical to give users some time during the test session. You give them their task and let them try. Don’t ask questions, and don’t give them hints. Be silent and observing.

Here is where the users get to make their observations and ask their questions. We encourage them to think aloud. If you have a user making comments on something worth exploring a little bit more, you can jump in and ask. Try and guide them from more abstract observations to more concrete and tangible ones.

Leading from abstract to concrete

Let me share an actual example from one of our more recent testing sessions. We had a user making comments on the general visual design. This user had a comment stating that

“this webpage seems to be frivolous, even childish.”

Right follow-up questions are then aimed at making this observation more concrete.

• In what sense you find this website to be frivolous and silly?
• Is it the graphic elements that are frivolous?
• What is it that makes your frivolous and childish?
• How bad is that?
• If you have the chance to do it in a different way, what would you do?
• What kind of changes would you make?

Listing all possible questions here would an impossible feat. But as a general rule – follow the idea that you should not guess what was user thinking when they said something. If it is too vague or abstract, guide them towards concrete, tangible, and relevant answers.

You should never say something like, “Oh, I get – what you wanted to say was…”

Don’t assume. Ask. Guide.

Closing questions

This is your opportunity to wrap-up the session. Do it wisely. Don’t rush it! Make a connection to those first questions you asked.

When I asked you, at the beginning of this session, about the similar products you used before – would you say that this product meets those expectations?

More importantly, assess the experience of the user. Ask questions about their perceived difficulty of the tasks they performed.

When I explained to you what are you going to do and explained your tasks – you had some feelings about how difficult it will be to carry out those tasks. Comparing your expectations then and your experience now – would you say that this was more difficult, easier or about the same as you expected?

Next, go on and assess their own affinity towards the product you just tested. Feel free to ask open-ended questions.

• Would you personally use this product?
• Why would you use it? / Why you wouldn’t use it?
• What are the situations where you see yourself using this product?

Then, try and see how well they can summarize the product. This will uncover parts of the product that made an impact on them. It will also give you a chance to learn about their emotions about the product itself.

If you had only three words at your disposal to describe this product, which words would you use?

After the summation, go on and assess the perceived benefits of the product. Give them a chance to tell you what would they change, given the opportunity.

Of all the features you saw, what would be the most beneficial for you? Why?
If you had the opportunity to change this product in any way you want, what would you change?

Our favorite question for usability testing

And, in the end, one of our favorite questions! Go try and put a user in a different situation. So far, they’ve been in the room with you, you established rapport, you seem to understand each other. Now, when this trust has been built, go on and ask them a quite insightful question.

If you had the chance to talk to a person who created (envisioned, designed, developed) this product, what would you say to that person?

This question can reveal a treasure trove of insights. The user will imagine a situation where they can face the person responsible for the product. It might make them a bit uncomfortable at the beginning, but this role-playing situation might help you a lot.

And lastly, as a typical summary – you can ask the user to rate different aspects of the product on a scale from one to seven. Why from one to seven? Because the range from one to five might be too reductionist. If they pick three, they are right in the middle. However, users need a bit more control and more granular scales. Also, using the scale from one to five would result in higher statistical uncertainties.

Besides this, you might want to research and see if using the SUS (system usability scale) would make sense. Then there is Likert’s scale and many other options at your disposal.

Let’s connect

If you are looking to develop your career as a usability and user researcher – we might be a good fit. Check out Careers page, and apply.

If you are a potential client, looking for professional and quality usability testing and user research services – we should talk. Users, their needs and usability of products and services is something we take a lot of pride in. We can be user advocates, while – at the same time – we understand your business environment and expectations.

Either way, I hope you found our post about the best questions for usability testing to be useful and insightful.