How to Find, Structure, and Format a Research Questionnaire

In this section, we will focus on the key considerations when designing a research questionnaire for a study.

• The first and foremost consideration is whether the scale used is reliable and valid. Simply finding a scale on the internet, such as on someone’s blog or educational website, does not guarantee its reliability and validity unless it has been properly tested and published in a reputable research journal.
• It is preferable to use a reliable and valid scale that has been published in a peer-reviewed international journal indexed by reputable databases like Master Journalist. Next, we need to determine the number of items to include in the questionnaire. For example, if we are measuring job satisfaction, it is generally recommended to include 4 to 6 items.
• This recommendation takes into account the use of structural equation modeling for data analysis. In structural equation modeling, items may be deleted during the analysis, resulting in a reduced number of items. By including 4 to 6 items from the start, we can anticipate potential item deletions and avoid analysis issues.
• Another consideration is whether to measure a construct at a lower or higher order. For instance, when measuring organizational commitment, we can choose to measure it as a higher order construct with sub-dimensions such as continuous commitment, normative commitment, and effective commitment. Alternatively, we can measure it as a lower order construct using 4 to 6 items. Different scholars may have different preferences, but generally lower order constructs are preferred unless the research requires a more complex model or higher order constructs are explicitly relevant.
• Matching the conceptualization of the constructs is another crucial aspect. Many students make the mistake of not clearly defining the concepts they are studying in their research. It is essential to define and conceptualize variables such as X, Y, and Z before selecting the questionnaire items. The conceptual scope of these variables determines their applicability in the study. Therefore, it is important to search for existing definitions of the concepts and ensure that the questionnaire items align with the conceptualization.
• Wording is also critical in questionnaire design. Students often use poorly worded questions, such as asking if respondents “like” or “love” their organization or if they “want to switch” jobs. To ensure the use of metric scales, it is recommended to phrase the questions as statements with response scales like “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” This allows for the use of methods like confirmatory factor analysis or regression analysis. Additionally, when using multiple items to measure a construct, the number of items and the response scale should be consistent.
• Overlapping between constructs is a common issue in social sciences research. For example, two concepts such as customer loyalty and word-of-mouth may have some overlap in their measurement items, even though they are conceptually distinct. Researchers must be cautious in selecting statements that accurately measure each construct to ensure discriminant validity.
• When searching for questionnaire items, it is crucial to refer to the original sources. Sometimes, researchers come across questionnaires in papers where they are provided in appendices or tables. However, it is important to verify if the authors of the paper actually developed the scale or if they adopted or adapted it from another source. It is recommended to go back to the original source to understand the scale’s development, including the number of items, response scale, and conceptualization.

These are the essential elements to consider when designing a questionnaire for a research study.