How to make a good screener
Every questionnaire or screener aims to allow respondents to answer accurately, totally, and without bias. In this article, common mistakes in screening questionnaire design are explained.
Relevant respondents are one of the essential elements of market research. Their quality depends on clear requirements and accurate recruitment. Companies usually use a screening questionnaire (a list of questions) to identify relevant respondents for their research, and they could sometimes meet different problems with respondents because of bad screening questionnaire design. An incorrect screener could decrease research quality significantly and lead to a loss of time and money for a company.
It is essential to prepare the correct screening questionnaire design to escape these problems. If a recruiter screens a potential respondent by phone or face-to-face, there is still a chance that a respondent may clarify unclear questions with a recruiter. However, respondents usually fill out a questionnaire by themselves, and they have no opportunity to ask somebody if they do not know how to answer. Due to their willingness to complete a survey, they may give false answers, and we, as researchers, will get bias which influences the whole research process. That is why it is important to pay close attention to screener design to save time and money for a company.
“High-quality screener is an essential tool for a recruiter.”
Tips for good screening questionnaire design
The aim of every questionnaire or screener is to allow respondents to answer accurately, totally and without bias. Also, a researcher should try to write questions and order them logically in order to create a situation of a daily conversation but not a research process. Respondents need to be relaxed and to answer naturally. Quality of a questionnaire could be influenced by different factors. It can be ambiguous and unclear questions. Bad order of questions, an order of codes within questions or inadequate response codes also may lead respondents to bias answers. There is no universal design template for a screening questionnaire, but we want to share some tips for good screening questionnaire design.
Yes/No vs. multi-Code questions (questions with several codes for response)
If a researcher needs to know what a respondent did or bought, for example, in the past 12 months it’s a bad option to ask “did you buy PRODUCT in the past 12 months?” and offer codes like YES and NO. More YES responses will be chosen than there are. Survey Sampling International shows in its study that there is a better way to figure out if a respondent buys your product. Better to design a multi-code question with several codes for response and set your product as one of the options. For example, the question “did you buy PRODUCT in the past 12 months?” turns into “what did you buy in the past 12 months?” and offers codes with names of different products including your product. Thereby, multi-code questions help to get more accurate information in comparison with Yes/No questions.
Do not allow missing answers/codes
What does it mean? It is a situation when one or several codes for response are missed, and respondents choose codes which are close to their opinion or experience. Missing codes are dangerous because it is difficult to see the real picture of a situation or attitudes. The only way to escape this mistake is to double-check and ask colleagues to take a fresh look.
Remember about effects of codes’ order
Also, it is possible to get biased results designing bad codes’ order. Order of codes for response may influence respondents’ answers in case of presence main category and sub-categories simultaneously (Question: where do you live? Codes: NSW, Sydney, VIC, e.g.) or codes with very similar meaning. If research design requires such kind of questions, similar codes should be kept close together, and sub-categories should be first and defined very clearly. Setting similar codes together helps to avoid false responses and biased results.
Do not forget “none of these” or “Don’t know” codes
Missed codes “none of these” or “don’t know” force respondents to give a false answer. A researcher will get biased data, and a recruiter will waste time while double-screening this respondent. The only way to escape this mistake is not to forget to include these codes.
Be careful with instructions in the question description
Sometimes respondents are asked to choose several options from a list. It may be products, touristic places or favourite actors. There is an opinion about this type of question that a researcher should set a preferable number of answers to choose from. So, respondents might be asked to choose up to 3 options or set a particular number. However, the study of the company Survey Sampling International shows that the best way is to allow respondents to pick as many codes as they want if a researcher does not have a specific reason to set a required number of codes for selection. Let respondents be free in their choice but do not allow blanks.
Every screener depends on research purposes and requirements. These, however, are common tips for good screening questionnaire design. They help to develop strong and effective recruitment tool which is essential for every recruitment process. A researcher or recruiter should pay high attention to their selection because relevant respondents play an important role in the achievement of perfect results and bringing benefits to a company.