Online surveys are a widely used mode of data collection. However, as no interviewer is present, respondents face any difficulties they encounter alone, which may lead to measurement error and biased or (at worst) invalid conclusions. Detecting response difficulty is therefore vital. Previous research has predominantly focused on response times to detect general response difficulty. However, response difficulty may stem from different sources, such as overly complex wording or similarity between response options. So far, the question of whether indicators can discriminate between these sources has not been addressed. The goal of the present study, therefore, was to evaluate whether specific characteristics of participants’ cursor movements are related to specific properties of survey questions that increase response difficulty. In a preregistered online experiment, we manipulated the length of the question text, the complexity of the question wording, and the difficulty of the response options orthogonally between questions. We hypothesized that these changes would lead to increased response times, hovers (movement pauses), and y-flips (changes in vertical movement direction), respectively. As expected, each manipulation led to an increase in the corresponding measure, although the other dependent variables were affected as well. However, the strengths of the effects did differ as expected between the mouse-tracking indices: Hovers were more sensitive to complex wording than to question difficulty, while the opposite was true for y-flips. These results indicate that differentiating sources of response difficulty might indeed be feasible using mouse-tracking.

中文翻译：

---

检测在线调查中的受访者负担：问题难度的不同来源如何影响光标移动

在线调查是一种广泛使用的数据收集方式。然而，由于没有访谈员在场，受访者独自面对他们遇到的任何困难，这可能会导致测量错误和有偏见或（最坏的情况）无效的结论。因此，检测响应难度至关重要。以前的研究主要集中在响应时间上以检测一般响应难度。然而，回答困难可能源于不同的来源，例如过于复杂的措辞或回答选项之间的相似性。到目前为止，指标是否可以区分这些来源的问题尚未得到解决。因此，本研究的目标是评估参与者光标移动的特定特征是否与增加回答难度的调查问题的特定属性相关。在预先注册的在线实验中，我们操纵了问题文本的长度、问题措辞的复杂性以及问题之间正交的回答选项的难度。我们假设这些变化将分别导致响应时间、悬停（移动暂停）和 y 翻转（垂直移动方向的变化）增加。正如预期的那样，每次操纵都会导致相应措施的增加，尽管其他因变量也受到影响。然而，鼠标跟踪指数之间的效果强度确实有所不同：悬停对复杂措辞比对问题难度更敏感，而 y 翻转则相反。这些结果表明，使用鼠标跟踪来区分反应困难的来源确实是可行的。