Screening Questions For Surveys

Screening questions are essential in surveys to identify the right respondents and exclude those who do not meet the criteria. By implementing them in your own surveys, you can ensure the quality and relevance of your data while respecting the time of potential participants.

But before implementing screening questions, you'll want to consider several factors such as their impact on your required sample size, the placement of these questions, and best practices. Here are some detailed guidelines to help you use screening questions effectively. Let's get started...

Screener question examples

There are two main types of screening questions, behavioural screeners and industry-specific screeners:

Behavioural screeners

Behavioural screeners identify respondents based on their behaviours, such as leisure activities or spending habits. For example:

"How many times a week do you exercise?"

- Every day

- Two to three times a week

- Once a week

- A couple of times a month

- Never

For example, if your survey focuses on the habits of frequent exercisers, you might choose to include only those who exercise at least once a week.

Industry-specific screeners

These screeners help filter out respondents whose answers might bias your research due to their industry involvement. For instance:

"Do you or anyone within your family work in the following industries?"

- Food

- Fashion

- Sports

- Household goods

- None of these

For example, to avoid bias, you could disqualify respondents who select the sports option if your survey is about sports retail.

Best practices when using screening questions

Once you've decided on the types of screening questions, consider how and when to ask them:

Ask a screening question at the beginning

This ensures that unsuitable respondents do not waste their time or compromise your data quality. Placing screeners at the start benefits both parties.

Add page breaks appropriately

Use skip logic to route qualified respondents through your survey and disqualify the rest. Place a page break immediately after the last screening question to efficiently manage this process. For more details, check out our guides on basic and advanced skip logic.

Keep screeners simple

Limit answer options to half a dozen and keep questions clear and concise. Remember, screeners are just the gateway to the rest of your survey.

Minimise ‘Yes’ or ‘No’ answers

These can introduce bias as some respondents might tend to answer 'yes' to appear agreeable.

Use multiple screeners if necessary

Sometimes, determining respondent suitability requires more than one question. Don’t hesitate to use multiple screening questions for more reliable results.

...but use the minimum required

Aim for the minimum number necessary to effectively filter respondents without causing fatigue.

Reflect ethical considerations

It's important to address the ethical implications of screening questions. Ensure transparency with respondents about why they are being screened and guarantee that your questions are non-discriminatory. This fosters trust and promotes fair treatment.

Common mistakes and how to avoid them

Mistake: Overly complex questions.

Solution: Simplify language and focus on one concept per question.

Mistake: Too many screening questions.

Solution: Limit to essential questions to avoid respondent fatigue.

Mistake: Ambiguous questions.

Solution: Use clear and specific language to avoid misunderstandings and ensure respondents interpret questions correctly.

Mistake: Using leading questions.

Solution: Ensure questions are neutral and do not suggest a preferred answer to avoid biasing responses.

Mistake: Ignoring demographic relevance.

Solution: Tailor screening questions to match the demographic profile relevant to the survey's objectives to gather more precise data.

Mistake: Lack of pilot testing.

Solution: Conduct a pilot test with a small group to identify potential issues and refine questions before full deployment.

Mistake: Inconsistent response options.

Solution: Standardise response options across similar questions to make it easier for respondents to provide consistent answers.

Mistake: Not updating questions regularly.

Solution: Review and update screening questions periodically to ensure they remain relevant and effective for current research goals.

Examples of poor screening questions

To further illustrate, here are some ineffective screening questions and why they fail:

"Do you like our products? Yes/No" – This question is too vague and likely to bias responses as it doesn't provide enough detail.

"Have you bought from us in the last year? Yes/No" – This fails to capture enough detail about the frequency and nature of purchases.

Choosing how many people to survey

After finalising your screening questions, consider the distribution and the number of recipients needed to achieve your desired response count. Screening questions typically lower response rates, but this is acceptable as it enhances the quality of feedback. You might, however, need to increase the total number of survey invitations.

Survey incidence rate

To achieve a certain number of qualified responses, consider your survey’s incidence rate. The incidence rate is the percentage of individuals who qualify to take your survey, calculated as follows:

For example, if you conduct an incidence rate test by sending your screening questions to 100 people and 75 qualify, your incidence rate is 75%. This calculation helps estimate how many more people you need to survey to get the desired number of qualified respondents.

What if my incidence rate is very low?

Consider refining your screening questions or increasing your survey distribution to reach more potential respondents.

How to analyse your survey responses

Once you’ve collected your survey responses, distinguish between complete and total responses. Complete responses are those where respondents answered the entire survey, whereas total responses include everyone who answered at least one question, including those disqualified.

To focus on complete responses, filter your survey data to view only those who passed the screeners. For detailed instructions on using filters, visit our ‘how to use filtering tools’ support page.

Advanced screening techniques

Explore more sophisticated methods for screening respondents, such as using branching logic, or combining multiple screener questions to form a composite score.

Branching logic: Set up branching logic to route respondents based on their answers to previous questions.

Composite scores: Combine answers from multiple questions to create a more robust screening process.

Example real-world applications

Screener questions play a vital role in ensuring survey responses come from the most relevant participants, enhancing the accuracy and value of the insights gathered. Here are several real-world examples that highlight the benefits of using effective screening questions in various scenarios:

Gymnasium researching new equipment requirements

Consider a scenario where a company wants to survey regular gym-goers about new equipment. Effective screening questions ensure only frequent gym users participate, leading to more relevant feedback and better insights.

Healthcare study on chronic illness management

A healthcare organisation wants to understand the experiences of patients managing chronic illnesses. Screening questions filter out respondents without chronic conditions, ensuring insights come from the target group.

Tech company testing a new software feature

A tech firm wants to gather feedback on a new software feature from experienced users. Screening questions ensure only those who use the software daily are included, providing valuable, informed feedback.

Retailer studying purchasing behaviour

A retailer wants to explore purchasing habits of high-frequency shoppers. Screening questions identify and include only respondents who shop at least once a week, offering more accurate insights into their behaviour.

Education sector researching online learning

An educational institution seeks feedback on online learning platforms from current students. Screening questions exclude those not enrolled in online courses, focusing the survey on relevant users.

Financial services evaluating investment products

A financial services company aims to assess opinions on new investment products. Screening questions ensure the survey targets active investors, resulting in more pertinent and actionable feedback.

Travel agency investigating travel trends

A travel agency wants to understand the preferences of frequent travellers. Screening questions select respondents who travel at least twice a year, yielding insights that are directly applicable to the target market.

Using screening questions to improve data quality

Screening questions are invaluable for eliminating irrelevant or biased feedback, which can complicate and prolong your data analysis and reporting. By incorporating screening questions at the start of your survey, you can significantly enhance data quality and streamline the analysis process.

Screening questions examples are numerous and varied, tailored to fit the specific needs of your survey. Whether behavioural or industry-specific, effective screening questions help you gather the most relevant and accurate data, ensuring your survey's success.

Understanding what are screening questions and how to use them effectively can transform your survey outcomes. Properly implemented screener questions refine your respondent pool, leading to more reliable and insightful results.

Remember, the goal of survey screening is not just to gather data, but to gather the right data. By focusing on the quality of your respondents through well-crafted screening questions, you lay the groundwork for meaningful and actionable insights.

Key takeaways

1. Purpose of screening questions: Screening questions are essential in surveys to identify suitable respondents and exclude those who do not meet the criteria, ensuring the quality and relevance of the data collected.

2. Types of screening questions: There are two main types: behavioural screeners, which identify respondents based on their behaviours, and industry-specific screeners, which filter out respondents whose answers might bias the research due to their industry involvement.

3. Best practices: Place screening questions at the beginning of the survey, use skip logic and page breaks appropriately, keep questions simple, minimise 'Yes' or 'No' answers, and use multiple screeners if necessary to ensure reliability.

4. Writing effective screening questions: Use specific language, limit response options to half a dozen, avoid double-barrelled questions, and pilot test questions with a small group to ensure clarity and effectiveness.

5. Analysing responses: Distinguish between complete and total responses by filtering the data to view only those who passed the screeners. This helps in focusing on the relevant data for analysis.

6. Advanced techniques and benefits: Employ advanced screening techniques like branching logic and composite scores to improve the robustness of the screening process. Effective screening leads to more relevant feedback and better insights, enhancing the overall quality of survey data.

Author's note: this blog was originally published July 2020 and updated July 2024