The goal of this work is to develop a task-agnostic feature upsampling operator for dense prediction where the operator is required to facilitate not only region-sensitive tasks like semantic segmentation but also detail-sensitive tasks such as image matting. Prior upsampling operators often can work well in either type of the tasks, but not both. We argue that task-agnostic upsampling should dynamically trade off between semantic preservation and detail delineation, instead of having a bias between the two properties. In this paper, we present FADE, a novel, plug-and-play, lightweight, and task-agnostic upsampling operator by fusing the assets of decoder and encoder features at three levels: (i) considering both the encoder and decoder feature in upsampling kernel generation; (ii) controlling the per-point contribution of the encoder/decoder feature in upsampling kernels with an efficient semi-shift convolutional operator; and (iii) enabling the selective pass of encoder features with a decoder-dependent gating mechanism for compensating details. To improve the practicality of FADE, we additionally study parameter- and memory-efficient implementations of semi-shift convolution. We analyze the upsampling behavior of FADE on toy data and show through large-scale experiments that FADE is task-agnostic with consistent performance improvement on a number of dense prediction tasks with little extra cost. For the first time, we demonstrate robust feature upsampling on both region- and detail-sensitive tasks successfully. Code is made available at: https://github.com/poppinace/fade

这项工作的目标是开发一种与任务无关的特征上采样算子，用于密集预测，其中算子不仅需要促进语义分割等区域敏感任务，而且还需要促进图像抠图等细节敏感任务。先前的上采样算子通常可以在任一类型的任务中很好地工作，但不能同时在两种任务中工作。我们认为与任务无关的上采样应该在语义保留和细节描述之间动态权衡，而不是在两个属性之间存在偏差。在本文中，我们提出了 FADE，一种新颖的、即插即用的、轻量级的、与任务无关的上采样算子，通过在三个层面上融合解码器和编码器特征的资产：（i）在上采样中同时考虑编码器和解码器特征内核生成； (ii) 使用高效的半移位卷积算子控制上采样内核中编码器/解码器特征的每点贡献； (iii) 通过依赖于解码器的门控机制实现编码器特征的选择性传递，以补偿细节。为了提高 FADE 的实用性，我们还研究了半移位卷积的参数和内存高效实现。我们分析了 FADE 在玩具数据上的上采样行为，并通过大规模实验表明，FADE 与任务无关，在许多密集预测任务上具有一致的性能改进，几乎没有额外成本。我们第一次成功地在区域和细节敏感的任务上展示了强大的特征上采样。代码位于：https://github.com/poppinace/fade