Multimodal-information-based optimized agricultural prescription recommendation system of crop electronic medical records,Journal of Industrial Information Integration

当前位置： X-MOL 学术 › J. Ind. Inf. Integr. › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Multimodal-information-based optimized agricultural prescription recommendation system of crop electronic medical records
Journal of Industrial Information Integration ( IF 10.4 ) Pub Date : 2024-12-07 , DOI: 10.1016/j.jii.2024.100748
Chang Xu, Junqi Ding, Bo Wang, Yan Qiao, Lingxian Zhang, Yiding Zhang

Multimodal Crop Electronic Medical Records (CEMRs) contain complex information, including disease symptoms, crop conditions, environmental factors, and diagnostic prescriptions, making them crucial for intelligent prescription recommendations. However, effectively integrating complementary features from different CEMRs modalities has remained a key challenge. Current CEMRs research primarily focuses on unimodal data, and simplistic approaches like feature concatenation struggle to achieve in-depth cross-modal interactions. This study introduces a novel agricultural prescription recommendation model (named AgriPR) based on cross-modal multi-layer feature fusion. The model initially employs task-adaptive pre-trained BERT (TA-BERT) and ConvNeXt to encode text and image unimodal features respectively. Subsequently, it utilizes Bilinear Attention Networks (BAN) to bilinear features and combines them with bimodal encoding features for a multilayer fusion representation. Finally, a dual-layer Transformer performs re-interaction to emphasize key fused features, resulting in precise prescription recommendations. To evaluate AgriPR, we constructed a real CEMRs dataset containing 13 prescription categories from Beijing Plant Clinic. Experimental results demonstrate that AgriPR achieves outstanding performance, with a classification accuracy of 98.88 %, surpassing state-of-the-art models. Furthermore, the study compares and analyzes 8 encoder combinations, 6 feature fusion strategies, and 6 network layer configurations, highlighting the model's design advantages. Lastly, the model's adaptability was also tested with incomplete modality inputs (text-only or image-only) and missing information inputs (e.g., crop, environment, symptoms). The findings confirm AgriPR's practical applicability, providing a high-performance solution for agricultural management systems.

更新日期：2024-12-07

点击分享查看原文

点击收藏

阅读更多本刊新发论文