Much of what we remember is not because of intentional selection, but simply a by-product of perceiving. This raises a foundational question about the architecture of the mind: how does perception interface with and influence memory? Here, inspired by a classic proposal relating perceptual processing to memory durability, the level-of-processing theory, we present a sparse coding model for compressing feature embeddings of images, and show that the reconstruction residuals from this model predict how well images are encoded into memory. In an open memorability dataset of scene images, we show that reconstruction error not only explains memory accuracy, but also response latencies during retrieval, subsuming, in the latter case, all of the variance explained by powerful vision-only models. We also confirm a prediction of this account with ‘model-driven psychophysics’. This work establishes reconstruction error as an important signal interfacing perception and memory, possibly through adaptive modulation of perceptual processing.

我们记住的大部分内容并不是有意选择的结果，而只是感知的副产品。这就提出了一个关于思维结构的基本问题：感知如何与记忆相互作用并影响记忆？在这里，受到将感知处理与记忆持久性相关的经典建议（处理级别理论）的启发，我们提出了一种用于压缩图像特征嵌入的稀疏编码模型，并表明该模型的重建残差可以预测图像的编码效果进入记忆。在场景图像的开放记忆数据集中，我们表明重建误差不仅可以解释记忆准确性，还可以解释检索期间的响应延迟，在后一种情况下，包含由强大的仅视觉模型解释的所有方差。我们还用“模型驱动的心理物理学”证实了这一说法的预测。这项工作可能通过感知处理的自适应调制，将重建误差确立为连接感知和记忆的重要信号。