Critical reviews of a test's measurement validity are valuable scientific contributions, yet even strong reviews can be undermined by subtle problems in how evidence is compiled and presented to readers. First, if discussions of poor reporting practices by a test's users are interwoven with discussions about validity support for the test itself, readers can be inadvertently misled into impressions of the latter which are improperly conflated with the former. Second, test reviewers should give at least as much careful attention to a test's external validity as to its structural validity; test reviewers who prioritize factor analysis and internal consistency at the expense of discriminant and convergent validity can inadvertently mislead readers into perceptions of a test which are more negative or more positive than is warranted by the evidence overall. In this commentary, we aim to help test evaluators in crafting critical investigations of measurement validity. We use Higgins et al.'s (2024) review of the Reading the Mind in the Eyes Test (RMET; Baron-Cohen et al., 2001) as a basis for discussion. We argue that their otherwise impressive review went astray in the two ways described above. After considering both the psychometric evidence that Higgins et al. (2024) provided and the external validity evidence that they did not provide, we conclude that their recommendations that the RMET should be abandoned, and that most prior research findings based on it should be reassessed or disregarded, are unwarranted.

中文翻译：

---

强大的测量有效性审查如何误入歧途：看看 Higgins 等人 （2024） 和对未来以测量为重点的审查的建议


对测试测量有效性的批判性评论是有价值的科学贡献，但即使是强有力的评论也可能被证据如何编译和呈现给读者的微妙问题所破坏。首先，如果测试用户对不良报告实践的讨论与对测试本身的有效性支持的讨论交织在一起，读者可能会无意中被误导到后者的印象中，这些印象与前者被错误地混为一谈。其次，测试审查员至少应该对测试的外部效度给予与其结构效度一样多的仔细关注;如果测试审阅者优先考虑因子分析和内部一致性，而牺牲了判别和收敛效度，可能会无意中误导读者对测试的看法，这些看法比整体证据所保证的更消极或更积极。在这篇评论中，我们旨在帮助测试评估者对测量效度进行批判性调查。我们使用 Higgins 等人 （2024） 对 Reading the Mind in the Eyes Test（RMET;Baron-Cohen et al.， 2001）作为讨论的基础。我们认为，他们原本令人印象深刻的评论在上述两种方面误入歧途。在考虑了 Higgins 等人（2024 年）提供的心理测量证据和他们没有提供的外部有效性证据后，我们得出结论，他们关于应该放弃 RMET 的建议，以及基于它的大多数先前的研究结果应该重新评估或无视，这是没有根据的。