当前位置: X-MOL 学术Int. J. Comput. Vis. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Blind Multimodal Quality Assessment of Low-Light Images
International Journal of Computer Vision ( IF 11.6 ) Pub Date : 2024-10-16 , DOI: 10.1007/s11263-024-02239-9
Miaohui Wang, Zhuowei Xu, Mai Xu, Weisi Lin

Blind image quality assessment (BIQA) aims at automatically and accurately forecasting objective scores for visual signals, which has been widely used to monitor product and service quality in low-light applications, covering smartphone photography, video surveillance, autonomous driving, etc. Recent developments in this field are dominated by unimodal solutions inconsistent with human subjective rating patterns, where human visual perception is simultaneously reflected by multiple sensory information. In this article, we present a unique blind multimodal quality assessment (BMQA) of low-light images from subjective evaluation to objective score. To investigate the multimodal mechanism, we first establish a multimodal low-light image quality (MLIQ) database with authentic low-light distortions, containing image-text modality pairs. Further, we specially design the key modules of BMQA, considering multimodal quality representation, latent feature alignment and fusion, and hybrid self-supervised and supervised learning. Extensive experiments show that our BMQA yields state-of-the-art accuracy on the proposed MLIQ benchmark database. In particular, we also build an independent single-image modality Dark-4K database, which is used to verify its applicability and generalization performance in mainstream unimodal applications. Qualitative and quantitative results on Dark-4K show that BMQA achieves superior performance to existing BIQA approaches as long as a pre-trained model is provided to generate text descriptions. The proposed framework and two databases as well as the collected BIQA methods and evaluation metrics are made publicly available on https://charwill.github.io/bmqa.html.



中文翻译:


弱光图像的盲多模态质量评估



盲图像质量评估 (BIQA) 旨在自动准确地预测视觉信号的客观分数,已广泛用于监测弱光应用中的产品和服务质量,涵盖智能手机摄影、视频监控、自动驾驶等。该领域的最新发展以与人类主观评分模式不一致的单峰解决方案为主,其中人类视觉感知同时由多个感官信息反映。在本文中,我们提出了一种独特的低光图像盲多模态质量评估 (BMQA),从主观评估到客观评分。为了研究多模态机制,我们首先建立了一个具有真实低光失真的多模态低光图像质量 (MLIQ) 数据库,其中包含图像-文本模态对。此外,我们专门设计了 BMQA 的关键模块,考虑了多模态质量表示、潜在特征对齐和融合以及混合自监督和监督学习。广泛的实验表明,我们的 BMQA 在拟议的 MLIQ 基准数据库上产生了最先进的准确性。特别地,我们还构建了一个独立的单像模态 Dark-4K 数据库,用于验证其在主流单模态应用中的适用性和泛化性能。Dark-4K 的定性和定量结果表明,只要提供预训练模型来生成文本描述,BMQA 就可以实现优于现有 BIQA 方法的性能。拟议的框架和两个数据库以及收集的 BIQA 方法和评估指标已在 https://charwill.github.io/bmqa.html 上公开提供。

更新日期:2024-10-17
down
wechat
bug