Extracting polygonal buildings from high-resolution remote sensing images is a critical task for large-scale mapping, 3D city modeling, and various geographic information system applications. Traditional methods are often restricted in accurately delineating boundaries and exhibit limited generalizability, which can affect their real-world applicability. The Segment Anything Model (SAM), a promptable segmentation model trained on an unprecedentedly large dataset, demonstrates remarkable generalization ability across various scenarios. In this context, we present SAMPolyBuild, an innovative framework that adapts SAM for polygonal building extraction, allowing for both automatic and prompt-based extraction. To fulfill the requirement for object location prompts in SAM, we developed the Auto Bbox Prompter, which is trained to detect building bounding boxes directly from the image encoder features of the SAM. The boundary precision of the SAM mask results was insufficient for vector polygon extraction, especially when challenged by blurry edges and tree occlusions. Therefore, we extended the SAM decoder with additional parameters to enable multitask learning to predict masks and generate Gaussian vertex and boundary maps simultaneously. Furthermore, we developed a mask-guided vertex connection algorithm to generate the final polygon. Extensive evaluation on the WHU-Mix vector dataset and SpaceNet datasets demonstrate that our method achieves a new state-of-the-art in terms of accuracy and generalizability, significantly improving average precision (AP), average recall (AR), intersection over union (IoU), boundary F1, and vertex F1 metrics. Moreover, by combining the automatic and prompt modes of our framework, we found that 91.2% of the building polygons predicted by SAMPolyBuild on out-of-domain data closely match the quality of manually delineated polygons. The source code is available at https://github.com/wchh-2000/SAMPolyBuild.

中文翻译：

---

SAMPolyBuild：调整 Segment Anything 模型以进行多边形建筑物提取


从高分辨率遥感图像中提取多边形建筑物是大规模制图、3D 城市建模和各种地理信息系统应用的关键任务。传统方法在准确描绘边界方面通常受到限制，并且泛化性有限，这可能会影响其实际适用性。Segment Anything Model （SAM） 是一种在前所未有的大型数据集上训练的可提示细分模型，在各种场景中表现出卓越的泛化能力。在此背景下，我们提出了 SAMPolyBuild，这是一个创新框架，它使 SAM 适用于多边形建筑提取，允许自动和基于提示的提取。为了满足 SAM 中对象位置提示的要求，我们开发了 Auto Bbox Prompter，它经过训练可以直接从 SAM 的图像编码器功能中检测建筑物边界框。SAM 掩码结果的边界精度不足以进行矢量多边形提取，尤其是在受到模糊边缘和树形遮挡的挑战时。因此，我们使用额外的参数扩展了 SAM 解码器，以实现多任务学习以预测掩码并同时生成高斯顶点和边界图。此外，我们开发了一种掩码引导的顶点连接算法来生成最终的多边形。对 WHU-Mix 向量数据集和 SpaceNet 数据集的广泛评估表明，我们的方法在准确性和泛化性方面达到了新的水平，显著提高了平均精度 （AP） 、平均召回率 （AR）、交并比 （IoU） 、边界 F1 和顶点 F1 指标。此外，通过结合框架的自动和提示模式，我们发现 91.SAMPolyBuild 基于域外数据预测的建筑物多边形中有 2% 与手动描绘的多边形的质量非常匹配。源代码可在 https://github.com/wchh-2000/SAMPolyBuild 上获得。