Estimating the motion of objects in depth is important for behavior, and is strongly supported by binocular visual cues. To understand both how the brain should estimate motion in depth and how natural constraints shape and limit performance in two local 3D motion tasks, we develop image-computable ideal observers from a large number of binocular video clips created from a dataset of natural images. The observers spatio-temporally filter the videos, and non-linearly decode 3D motion from the filter responses. The optimal filters and decoder are dictated by the task-relevant image statistics, and are specific to each task. Multiple findings emerge. First, two distinct filter subpopulations are spontaneously learned for each task. For 3D speed estimation, filters emerge for processing either changing disparities over time (CDOT) or interocular velocity differences (IOVD), cues that are used by humans. For 3D direction estimation, filters emerge for discriminating either left-right or towards-away motion. Second, the filter responses, conditioned on the latent variable, are well-described as jointly Gaussian, and the covariance of the filter responses carries the information about the task-relevant latent variable. Quadratic combination is thus necessary for optimal decoding, which can be implemented by biologically plausible neural computations. Finally, the ideal observer yields non-obvious-and in some cases counter-intuitive-patterns of performance like those exhibited by humans. Important characteristics of human 3D motion processing and estimation may therefore result from optimal information processing in the early visual system.Significance statement Humans and other animals extract and process features of natural images that are useful for estimating motion-in-depth, an ability that is crucial for successful interaction with the environment. But the enormous diversity of natural visual inputs that are consistent with a given 3D motion-natural stimulus variability-presents a challenging computational problem. The neural populations that support the estimation of motion-in-depth are under active investigation. Here, we study how to optimally estimate local 3D motion with naturalistic stimulus variability. We show that the optimal computations are biologically plausible, and that they reproduce sometimes counterintuitive performance patterns independently reported in the human psychophysical literature. Novel testable hypotheses for future neurophysiological and psychophysical research are discussed.

中文翻译：

---

使用自然刺激对局部深度运动进行最佳估计。


估计物体的深度运动对于行为很重要，并且得到双眼视觉线索的强烈支持。为了了解大脑应该如何深入估计运动以及自然约束如何塑造和限制两个局部 3D 运动任务中的性能，我们从自然图像数据集创建的大量双目视频剪辑中开发了图像可计算的理想观察者。观察者对视频进行时空过滤，并从滤波器响应中非线性解码 3D 运动。最佳过滤器和解码器由与任务相关的图像统计数据决定，并且特定于每个任务。出现了多个发现。首先，为每个任务自发学习两个不同的过滤器子群。对于 3D 速度估计，出现了过滤器，用于处理随时间变化的差异 （CDOT） 或眼间速度差异 （IOVD），这些都是人类使用的线索。对于 3D 方向估计，出现了用于区分左右或向外运动的滤波器。其次，以潜在变量为条件的过滤器响应被很好地描述为联合高斯，并且过滤器响应的协方差携带有关任务相关潜在变量的信息。因此，二次组合对于最佳解码是必要的，这可以通过生物学上合理的神经计算来实现。最后，理想的观察者会产生非明显的——在某些情况下是违反直觉的——表现模式，就像人类所表现出的一样。因此，人类 3D 运动处理和估计的重要特征可能是早期视觉系统中最佳信息处理的结果。意义陈述 人类和其他动物提取和处理自然图像的特征，这些特征对于估计深度运动很有用，这种能力对于与环境的成功交互至关重要。但是，与给定 3D 运动（自然刺激变化）一致的自然视觉输入的巨大多样性提出了一个具有挑战性的计算问题。支持深度运动估计的神经种群正在积极研究中。在这里，我们研究了如何利用自然刺激变化来优化估计局部 3D 运动。我们表明，最佳计算在生物学上是合理的，并且它们再现了人类心理物理学文献中独立报道的有时违反直觉的表现模式。讨论了未来神经生理学和心理物理学研究的新颖可检验假设。