A recent article in Perspectives on Psychological Science (Webb & Tangney, 2022) reported a study in which just 2.6% of participants recruited on Amazon’s Mechanical Turk (MTurk) were deemed “valid.” The authors highlighted some well-established limitations of MTurk, but their central claims—that MTurk is “too good to be true” and that it captured “only 14 human beings . . . [out of] N = 529”—are radically misleading, yet have been repeated widely. This commentary aims to (a) correct the record (i.e., by showing that Webb and Tangney’s approach to data collection led to unusually low data quality) and (b) offer a shift in perspective for running high-quality studies online. Negative attitudes toward MTurk sometimes reflect a fundamental misunderstanding of what the platform offers and how it should be used in research. Beyond pointing to research that details strategies for effective design and recruitment on MTurk, we stress that MTurk is not suitable for every study. Effective use requires specific expertise and design considerations. Like all tools used in research—from advanced hardware to specialist software—the tool itself places constraints on what one should use it for. Ultimately, high-quality data is the responsibility of the researcher, not the crowdsourcing platform.

中文翻译：

---

高质量在线数据收集的负担在于研究人员，而不是招聘平台


最近在《心理科学视角》（Webb & Tangney，2022）上的一篇文章报告了一项研究，其中只有2.6%的参与者在亚马逊的Mechanical Turk（MTurk）上被招募被认为是“有效的”。作者强调了 MTurk 的一些公认的局限性，但他们的核心主张——MTurk “好得令人难以置信”，并且它“只捕获了 14 个人......[出]N = 529“——具有根本的误导性，但已被广泛重复。本评论旨在 （a） 纠正记录（即，通过表明 Webb 和 Tangney 的数据收集方法导致数据质量异常低）和 （b） 为在线运行高质量研究提供视角的转变。对 MTurk 的消极态度有时反映了对平台提供的内容以及应该如何在研究中使用它的根本误解。除了指出详细介绍 MTurk 有效设计和招募策略的研究外，我们还强调 MTurk 并不适合每项研究。有效使用需要特定的专业知识和设计考虑。与研究中使用的所有工具一样，从高级硬件到专业软件，该工具本身对人们应该使用它的目的施加了限制。归根结底，高质量的数据是研究人员的责任，而不是众包平台的责任。