Leading Questions in Surveys

Bias can occur quite frequently in our everyday lives. And we are often the instigators, using bias to turn a situation in our favour. Social desirability bias is a good example of this. In this scenario, we will mask our true selves and present another version of our ourselves that we believe others will find more favourable in order for them to accept us.

However, there are also occasions when we definitely don’t want to introduce any bias whatsoever.

Consider what you would do if you were asking someone for some important information as part of a normal conversation. It’s likely you would try to keep your question as straightforward and neutral as you could, to get the best, most honest and valuable feedback. You would certainly be careful about injecting any opinions into your question, to avoid risking a potentially biased response.

Well, the same can be said with the questions you create and ask through your survey. If you ask leading questions, you’ll risk biased answers, diluting the value of your survey and damaging what you’re able to do with the insight you’ve gained.

So, what exactly do we mean by leading questions? What types of leading questions do you need to look out for? What are the main disadvantages of this question type? And how can you avoid leading questions in your survey?

We’ll explore all these areas in this blog, but first of all we need to define leading questions.

What are leading questions?

When we refer to leading questions, we’re essentially talking about a question type that prompts a respondent towards providing an already-determined answer. Typically the question is framed in such a way that it implies or points respondents to its answer(s). In this way, it’s helping to confirm information already contained in the question, rather than providing a true and an unbiased answer.

Organisations sometimes use them to hasten the speed it takes to persuade a customer to make a particular decision. For example, if you were selling an insurance plan that you wanted clients to sign up for, you could potentially speed up this process with a leading question such as:

“When would you like to sign up for our insurance plan?

Such directness is likely to be more effective than simply asking a client if signing up for your insurance plan would be of interest to them.

Types of leading questions

When it comes to types of survey questions, leading questions are categorised based on their inherent intent, objective and framing. This means they have the potential to change the direction of your survey results. Therefore, it’s useful to know what types of leading questions to look out for.

Subsequently, we’ve outlined the key types to be aware of, along with examples of leading questions in questionnaires.

Leading questions that are based on assumptions

An assumption-based leading question communicates a preconceived notion and is therefore framed as such. In other words, the entire question is based on the assumptions of the interrogator, with regards to the actions or perception of the client.

They can frequently come up in feedback surveys, where the survey creator is trying to evaluate a respondent’s perception of a product, service or process.

For example, when working on customer survey questions, to gather feedback about a particular product or service, you could include an assumptive leading question such as:

“How much did you enjoy using our product?

Here the question presupposes that the client must have enjoyed using the product. And it’s only interested in measuring the degree of that enjoyment.

Leading questions based on interconnected statements

This type of leading question contains two interrelated or closely connected statements. With the interconnected leading question, the first statement typically provides a wider but slightly biased context to what the survey creator wants the respondent to answer in the second statement.

A good example of this is a question you might find in an employee questionnaire, aimed at collecting useful information to help shape future workplace policy.

“Many employees enjoy working overtime. What do you think about this?

With this question an employer could get a better feel about staff feelings towards work overtime opportunities.

Leading questions based on a direct implication

Direct implication leading questions are ones that allow respondents to consider the potential results of their answer in terms of how it might shape something in the future.

For example, event planners are continually looking for attendee feedback to help improve the success of future events. In such a scenario, they might ask the following question.

“If you enjoyed this event, should we host one in a similar manner?

Leading questions based on coerciveness

The objective of a coercive leading question, also known as a leading question with tags, is to try to force respondents to provide them with a specific answer; usually in the affirmative. Compared with the more subtle emphasis of other leading question types, coercive leading questions are much more forceful in tone.

So, if you were involved with surveys in healthcare, some examples of coercive leading questions you could include.

“You enjoyed your stay in hospital, didn’t you?

“You loved interacting with our medical staff, didn’t you?

“We satisfactorily met your healthcare needs with our medical expertise, didn’t we?

Leading and loaded questions – the key differences

Having discussed types of leading questions, it’s also worth mentioning loaded questions, as they can also lead to biased answers.

While there are some slightly different interpretations and understandings of these questions, there is one overarching difference that can help you to distinguish between the two. This becomes evident when you explore a definition of the two.

By the way they are phrased, leading questions intend to lead people to answer a question in a specific way. They typically do this by including information that they want confirmed, rather than digging deeper to try and get a true answer.

While loaded questions are similar to leading questions in that they subtly (or not so subtly) push users towards making a particular response, their key difference is in the assumptions they make about the respondent, which is included implicitly in the question.

A good example of this is evident in the following questions.

“Where do you enjoy drinking beer?

This question wrongly assumes that everyone answering drinks beer.

“How often do you exercise? Twice a day?

Similarly, this question assumes those answering it regularly exercise and typically do this twice a day.

To take the bias out of these statements, each question should require the respondent to provide qualifying information that’s relevant to them. Alternatively, you could employ skip logic to skip them past any non-relevant questions.

Leading and loaded questions have small differences, but it’s important to remember that they both help to confuse, mislead, or influences users into making a particular selection. So, you must try to avoid them as much as you can.

Disadvantages of leading questions

Having touched on loaded questions and their differences, we can now return to leading questions and the reasons why you should be careful about their use. Generally speaking there are many disadvantages to using leading questions, and it’s helpful to be aware of these.

Highly subjective responses

The problem with leading questions, is that the bias they introduce often results in highly subjective responses. And when the answers you gather no longer offer a true reflection of a respondent’s perception it can harm the quality of your data.

Nothing new in terms of insight

When you consider that the objective for a lot of research is to learn something new, leading questions can be quite detrimental to this process. Given that the response to leading questions is often a regurgitation of the biases communicated in the questions, it makes it impossible to gather any fresh insights.

False feedback

Another major disadvantage resulting from leading questions is false feedback. So, if you’re trying to measure market perception of your product or service, false feedback is going to be disastrous in terms of any improvements you believe you should make as a result of this.

Respondents may abandon your survey

It’s also prudent to be aware of how leading questions may affect respondents completing your survey. A biased question may leave some respondents feeling uncomfortable. And as well as not wanting to answer that question, they may decide to abandon your survey altogether. Too many respondents leaving your survey can harm your response rate and the validity of your data.

How to avoid leading questions

Having examined some of the problems with leading questions, you’ll want to do everything you can to avoid inadvertently slipping them into your questionnaire.

Some useful tips to help you include.

Check questions with someone removed from your project

When you’re creating questions for your survey, it can be helpful to have them checked over by a fresh pair of eyes. It’s easier for someone not involved in your project or the topic you’re researching to spot potentially biased questions than you could.

Keep questions clear and simple

Try to keep your questions as simple and straightforward as possible. Try to avoid leading respondents down a particular answer route. Instead offer appropriate answer options and offer the “other” option to make sure your survey is easy to respond to.

Avoid jargon or difficult to understand terminology

To help respondents to answer your questions, you want to avoid using jargon, technical or difficult to understand language. This can otherwise result in leading questions, where recipients choose only the answer option that is best understood by them.

Non leading question examples

Once you have some ideas about how to avoid leading questions, it can be useful to have some examples of non-leading questions to draw on. It can be even more valuable to outline a leading question first, followed by the fix to make it non-leading.

Here’s some examples to get you thinking about this.

Leading question: “How amazing is our hard-working customer service team?

Fixed to non-leading: “Describe your experience with our customer service team?

Leading question: “How great would you say our product is?

Fixed to non-leading: “How would you rate our product?