The diversity of building functions is vital for urban planning and optimizing infrastructure and services. Street-view images offer rich exterior details, aiding in function recognition. However, street-view building function annotations are limited and challenging to obtain. In this work, we propose a geometry-aware semi-supervised method for fine-grained building function recognition, which effectively uses multi-source geoinformation data to achieve accurate function recognition in both single-city and cross-city scenarios. We restructured the semi-supervised method based on the Teacher–Student architecture into three stages, which involve pre-training for building facade recognition, building function annotation generation, and building function recognition. In the first stage, to enable semi-supervised training with limited annotations, we employ a semi-supervised object detection model, which trains on both labeled samples and a large amount of unlabeled data simultaneously, achieving building facade detection. In the second stage, to further optimize the pseudo-labels, we effectively utilize the geometric spatial relationships between GIS map data and panoramic street-view images, integrating the building function information with facade detection results. We ultimately achieve fine-grained building function recognition in both single-city and cross-city scenarios by combining the coarse annotations and labeled data in the final stage. We conduct extensive comparative experiments on four datasets, which include OmniCity, Madrid, Los Angeles, and Boston, to evaluate the performance of our method in both single-city (OmniCity & Madrid) and cross-city (OmniCity - Los Angeles & OmniCity - Boston) scenarios. The experimental results show that, compared to advanced recognition methods, our method improves mAP by at least 4.8% and 4.3% for OmniCity and Madrid, respectively, while also effectively handling class imbalance. Furthermore, our method performs well in the cross-categorization system experiments for Los Angeles and Boston, highlighting its strong potential for cross-city tasks. This study offers a new solution for large-scale and multi-city applications by efficiently utilizing multi-source geoinformation data, enhancing urban information acquisition efficiency, and assisting in rational resource allocation.

中文翻译：

---

通过几何感知半监督学习，使用街景图像和 GIS 地图数据进行精细的建筑功能识别


建筑功能的多样性对于城市规划和优化基础设施和服务至关重要。街景图像提供丰富的外部细节，有助于功能识别。但是，街景建筑物功能注释有限且难以获取。在这项工作中，我们提出了一种用于细粒度建筑功能识别的几何感知半监督方法，该方法有效地利用多源地理信息数据来实现单城市和跨城市场景下的准确功能识别。我们将基于师生架构的半监督方法重组为三个阶段，包括建筑立面识别、建筑功能标注生成和建筑功能识别的预训练。在第一阶段，为了实现有限标注的半监督训练，我们采用了半监督目标检测模型，同时对标记样本和大量未标记数据进行训练，实现建筑立面检测。在第二阶段，为了进一步优化伪标签，我们有效利用 GIS 地图数据与全景街景图像之间的几何空间关系，将建筑功能信息与立面检测结果相结合。我们通过在最后阶段将粗标注和标注数据相结合，最终实现单城市和跨城市场景下的细粒度建筑功能识别。我们对四个数据集进行了广泛的比较实验，其中包括OmniCity、马德里、洛杉矶和波士顿，以评估我们的方法在单一城市（OmniCity和马德里）和跨城市（OmniCity - 洛杉矶和OmniCity - 波士顿）场景中的表现。 实验结果表明，与先进的识别方法相比，我们的方法分别将 OmniCity 和 Madrid 的 mAP 提高了至少 4.8% 和 4.3%，同时还能有效地处理阶级不平衡。此外，我们的方法在洛杉矶和波士顿的交叉分类系统实验中表现良好，突出了其在跨城市任务中的巨大潜力。本研究通过高效利用多源地理信息数据，提高城市信息获取效率，协助合理配置资源，为大规模、多城市应用提供了新的解决方案。