Usability Testing: How Surveys Can Improve It

It might sound obvious, but if you’re to build a successful product or service, you either need to be solving some sort of problem or meeting a specific user need.

There are a number of methods that can help you identify what you need to know including usability testing. But it may not always be the best or most efficient approach for you to start with, it all depends on what you want to learn.

For instance, while usability testing is ideal when you need more detailed feedback about what users struggle with and why, sometimes you’ll want to conduct a more a top-level exploration of the audience for your product or service. In this scenario, you might be more focused on what they want, and what they think – and you’ll require a greater volume of feedback to put these answers into perspective.

This is where surveys come to the fore. While most people will be familiar with surveys, marketing teams have been making great use of them long before usability and UX became common practices. However, surveys are now being used to complement usability research efforts, as they are great for answering questions about who your users are, what they think of your products, and what issues they’re encountering.

In this blog, we’ll take a look at how you can combine surveys with usability testing to get more from your research efforts. But before we do that, we’ll talk a little bit more about usability testing, for those who are less familiar with the concept.

What is usability testing?

Essentially, usability testing seeks to understand how easy it is for your product or service to be used by its audience. And to achieve this, you need to be asking yourself questions such as:

Can our audience actually use our product/service features?

Can our audience understand how to use our product/service features to meet their goals and solve their problems?

For example, consider a new design for a door handle that looks like a square piece of metal at the centre a door. At this point, you’re likely to have a number of questions in mind that you’ll want answered, which could include the following:

Will those who wish to open that door understand how to use the handle?

Will they look to twist it, pull it, or push it?

If they understand how to manipulate it, can they successfully physically interact with it, or is it too large and sharp to turn with their bare hands?

Of course, it’s unlikely that a square door handle will be a usable product, but it’s helpful to illustrate how usability testing can help designers and product owners get answers to better meet the needs of their users.

According to UX experts, Nielsen Norman Group, there are five quality components of usability, which we’ll detail in turn using the door handle example.

Learnability

Under the learnability component, the usability test will be looking to determine how simple it was for users to accomplish basic tasks on first encountering the new design.

The square door handle is unlikely to perform well under this criterion, since it’s not familiar or intuitive for the user base.

Efficiency

Under the next criterion of efficiency, following their interaction with the design, the focus would shift to measuring how quickly they could perform tasks.

The hope here is that if they’re able to physically turn the door handle the target audience should be able to open the door pretty quickly.

Memorability

Under the memorability criterion, users are tested on how easily they can re-establish proficiency with the design, after a period of not using it.

The testers will be keen to see if users remember how to approach the door with the unusual handle the next time, they enter this room?

Errors

This criterion is all about measuring the number of errors users make, their severity, and how easily users can they recover from them.

A door that they can’t open, a sore hand or a deep sense of frustration might hamper the overall experience for the user, that might take a lot of time for them to recover from.

Satisfaction

The last criterion looks to ascertain how satisfied users find it interacting with the design.

Although, this criterion is rather subjective, if something is as confusing and uncomfortable to use as a large square door handle, it’s unlikely to be a pleasant experience.

The process of running a usability test involves getting the actual users of a product or service to interact with it, so they can assess how usable they find it.

Usability testing methods typically include asking the test subjects to complete a set of standardised, specific tasks while UX designers watch, listen and take notes or record what happens.

For instance, a website usability test might involve a user completing tasks such as clicking through website pages, submitting landing page forms and testing calls-to-action (CTAs). Following this they may be asked to complete tasks that are more representative of realistic scenarios in a typical user journey. These could include tasks such as making a purchase on an e-commerce website or trying to locate a specific piece of information such as a company address or product price.

Whether you use simple, specific tasks or complex flows, the aim of a usability test is to highlight problems, collect quantitative and qualitative data, and determine how usable your design is once it’s being used by your target audience.

The difference between usability testing and surveys in the research environment

Even if you’ve not performed usability testing before, you should be able to see from what we’ve just outlined, just how detailed the captured data is under this approach.

However, it’s not always the most efficient method for your research budget and the hours you have available. Sometimes you need a higher volume of feedback to the questions you’re asking, than is practical for a usability study. It's all about determining what you want to learn.

To highlight the difference between using a survey or a usability test for your research, consider these different methods during a trip to the beach.

In the first scenario, you’re on a sheltered side of the beach, where when the tide goes out, you can walk for hours and see hundreds of small creatures in the shallow pools of the tide. Although they’re not doing very much, you can see a wide variety of sea life. By contrast, you can also go snorkelling or scuba diving to see those fish in action. This is where you can also observe them interacting with their environment and learn more about their behaviours.

Taking these two different scenarios, your survey is like the long walk among the tide pools that gives a top-level view of the creatures, while your usability test is a shorter but deeper dive beneath the water to see how these creatures interact with their surrounding environment.

Combining surveys with usability testing

If you’re to realise the greatest benefits from your research efforts, it can help to combine the use of surveys with usability testing, which can offer you further value throughout the research process from planning and implementation to follow-up.

Pre-test

Two of the biggest questions you’re likely to face as you start to plan a usability study are:

Who are my users?

What do I want to test? (which goes hand in hand with what you want to learn)

Fortunately, surveys can help you to get at this information.

Using surveys to learn more about your users

Product surveys in particular can help you to work out what you need to include in your study’s sample group, in terms of demographics such as gender, age, income and more.

When you’re gathering user data, you’ll hope to start seeing useful patterns in your data. To discover these patterns, you might want to ask users survey questions such as:

Why did you start using our product?

What do you hope to get from it?

What is your profession? (to help you gauge whether there are any connections between their professions and the product)

What are your skills or interests? (to help you identify whether these are relevant to how they might potentially use the product)

In gathering this sort of data, you’ll want to capture random actual users or potential users as your survey respondents.

Using surveys to learn what to test

You can also use surveys in your research to identify where your product’s pain points are, through asking questions such as:

Which of these areas do you struggle with the most while using our product?

or

How would you rate your experience with our checkout process?

This can give you greater clarity about where you need to focus your usability test.

During or post usability test

Using surveys to help quantify usability insights

To help put your findings in to perspective when you’re carrying out your usability test, you may want to collect some additional quantitative responses.

This can give you greater insight into a number of areas including:

• Users’ perceptions about the severity, or level of frustration with certain issues to do with your product
• Any parts of your product which are perceived by your users as more problematic
• Users’ overall perception of the product

You can also use surveys to capture background information on your participants including more detailed demographics, hobbies, or preferences, to help better interpret their responses.

Following up with surveys to measure the success of usability testing

After you’ve completed your usability research, you’ll be keen to start making improvements based on the findings you’ve learned.

At this point surveys can help you to benchmark the results of your efforts, as you should know where your problems are. Then, as you continue to make improvements, you can use surveys to measure your results. Of course, you'll also want to follow up with usability testing, because while a survey gives you that top level view of what users are thinking, usability testing is better for monitoring their user interactions.

Concluding thoughts

We hope you found this blog interesting, and if you’re already using usability testing, you can see how combining it with surveys offers further value for your research.

The key thing to remember is that every research method has its strengths and weaknesses. So, if you can combine a number of different approaches, it will give you an all-round perspective that not only strengthens your research but delivers greater value for those who need and will use its data.