Tables and figures are usually used to present information in a structured and visual way in scientific documents. Understanding the tables and figures in scientific documents is significant for a series of downstream tasks, such as academic search, scientific knowledge graphs, and so on. Existing studies mainly focus on detecting figures and tables from scientific documents, interpreting their semantics, and integrating them into downstream tasks. However, a systematic and comprehensive literature review on the mining and application of tables and figures in academic papers is still missing. In this article, we introduce the research framework and the whole pipeline for understanding tables and figures, including detection, structural analysis, interpretation, and application. We deliver a thorough analysis of benchmark datasets, recent techniques, and their pros and cons. Additionally, a quantitative analysis of the effectiveness of different models on popular benchmarks is presented. We further outline several important applications that exploit the semantics of scientific tables and figures. Finally, we highlight the challenges and some potential directions for future research. We believe this is the first comprehensive survey in understanding scientific tables and figures that covers the landscape from detection to application.

表格和图形通常用于在科学文献中以结构化和可视化的方式呈现信息。理解科学文献中的图表对于学术检索、科学知识图谱等一系列下游任务具有重要意义。现有的研究主要集中在从科学文档中检测图形和表格，解释它们的语义，并将它们集成到下游任务中。然而，目前还缺乏对学术论文中图表的挖掘和应用进行系统、全面的文献综述。在本文中，我们介绍了研究框架和理解表格和图形的整个流程，包括检测、结构分析、解释和应用。我们对基准数据集、最新技术及其优缺点进行了全面分析。此外，还对不同模型在流行基准上的有效性进行了定量分析。我们进一步概述了利用科学表格和图形语义的几个重要应用程序。最后，我们强调了未来研究的挑战和一些潜在方向。我们相信，这是理解科学表格和图表的第一个全面调查，涵盖了从检测到应用的整个过程。