How To Analyse Open Ended Questions

When it comes to your research and what you’re trying to find out, the research process behind it will generally fall into two camps quantitative or qualitative.

While quantitative research is more focused on the numbers, such as the volume of your audience that holds a particular view, qualitative research would be more concerned about identifying exactly what those people felt. Subsequently, this will influence the survey questions that you use with quantitative data typically collected by asking closed ended questions, and the qualitative approach more focused on open ended questions. But you can use a mix of the two in your questionnaire.

For the purposes of this blog, we will be focusing on open ended questions in surveys and how best to analyse them. But before we look at the analysis side of open ended questions, it helps to explore in more detail what they are and how they differ from closed ended questions.

Difference between open and closed questions

The main difference between closed and open ended questions can be seen in terms of the flexibility respondents are offered for answering them.

With closed ended questions, respondents can only answer ‘yes’ or ‘no’ or choose their answer from a pre-set number of multiple-choice formats and options. This makes the data quicker and easier for researchers to review and analyse.

In contrast, with open ended questions respondents are given a free text box and the freedom to answer the question in their own words. While the answers to open ended questions can be slower and less simple to analyse, they do offer researchers more detailed and valuable information. This is because the answers provide more insight into why respondents have answered in the way that they have and their feelings behind this. In addition, open ended questions can often deliver insights that you didn’t expect, which can be beneficial in providing you with ideas that you may never have otherwise considered.

It’s worth noting that there are now some analysis tools available, which make it simpler and quicker to uncover the valuable insight provided by open ended questions, which we will look at next.

How to analyse open ended questions in a survey

When it comes to your survey, you’ll want to get as much value from your results as you can. However, respondents who don’t meet your target criteria, only answer a portion of your survey, or provide nonsensical feedback can easily harm the quality and reliability of your data.

Therefore, before analysing open ended questions, using analysis tools to review your data, it helps to remove any unhelpful data from your survey results first.

Once you’ve done this, you’re ready to start looking at analysis tools.

Text analysis

In terms of tools to help analyse and uncover the most valuable respondent sentiment in open ended questions, the Text Analysis tool, is probably one of the most effective around.

Text analysis is essentially a machine learning technique, which can be used to automatically extract valuable data insights from unstructured text data. Text analysis is usually available with a range of options that you can select to run the tool and comb through your open text responses, looking for correlations and keywords.

Options to run text analysis

When it comes to analysing open ended questions with the text analysis tool, you can either set up and apply categories, or quickly view responses as a Word Cloud or Word List.

With any research there’s usually a number of different outcomes you think may happen. And therefore, if you’re running a survey, there’s likely to be some themes, patterns and correlations in the responses that you’ll be hoping you might see as supporting evidence for one of these outcomes. This is where the categories option and the tagging of text within responses can be useful.

For example, if you were running a product testing survey for a new website and wanted to know how positively or unfavourably a sample of your customer base viewed it in terms of its usability, functionality, engagement and more, the categories option could help you with this.

In such a scenario, you might choose to set up two categories, one for positive phrases and the other for negative phrases and tag key words and phrases you wanted the text analysis tool to search for under each category. So, in the case of positive phrases, you might want to scan for words and phrases such as, ‘easy’, ‘easy to use’, ‘user friendly’, ‘simple to navigate’, ‘great user interface’, ‘great UX’ and more. You could also apply different colour codes for positive and negative words, to make the data simpler to review at the analysis stage.

Having run the text analysis over your completed responses, the tool will count the total number of tagged words and phrases to each question. It will then display these as a pie chart, so it’s easy to see how favourably or unfavourably respondents view different aspects of your website. This would help you to identify which areas customers were happy with and which areas you needed to improve.

If you don’t fancy using the categories feature, there are some further options you can use with the text analysis tool including the Word Cloud and Word List features.

By selecting the Word Cloud option, it will provide you with an image of the most common words used in the responses to your questions. So, if you were asking respondents a question about how they perceived the service of your support team, it could generate a Word Cloud image similar to the one below.

Alternatively, you could select the Word List option, which would give you a quick breakdown of these words and how many times they were mentioned, from the most to the least used.

To learn more about text analysis and how you can set it up, you might want to take a look at the guide we’ve put together to help with this.

Final thoughts

Whether you’re new to open questions or have been already using them for some time, we hope this overview of the question type and advice on how to analyse open ended questions has been useful to you.

Open ended questions are hugely beneficial in providing you with a deeper insight into the thoughts and feelings of your respondents, that would be difficult to achieve with any other question type. And fortunately, with tools such as text analysis you can now get to crux of the detail you need quicker and more easily than ever before.