Advancements in deep learning tools originally designed for natural language processing are also applied to applications in the field of process control. Transformers, in particular, have been used to leverage self-attention mechanisms and effectively capture long-range dependencies. However, these architectures require extensive data representative of a specific process, which is not always available. To address this issue, transfer learning has emerged as a machine learning technique that enables pretrained models to adapt to new tasks with minimal additional training. This paper demonstrates a process that combines transfer learning with transformer architectures to enable a data-driven approach to control tasks, such as system identification and surrogate control modeling, when data are scarce. In this study, large amounts of data from a source system are used to train a transformer that models the dynamics of target systems for which limited data are available. The paper compares the predictive performance of models trained only on target system data with models using transfer learning including a modified transformer architecture with a physics-informed neural network (PINN) component. The results demonstrate improved predictive accuracy in system identification by up to 45% with transfer learning and up to 74% with both transfer learning and a PINN architecture. Similar accuracy improvements were observed in surrogate control tasks, with enhancements of up to 44% using transfer learning and up to 98% with transfer learning and a PINN architecture.

中文翻译：

---

用于过程控制应用的物理信息迁移学习


最初为自然语言处理设计的深度学习工具的进步也适用于过程控制领域的应用。特别是 Transformers，已被用于利用自我注意机制并有效地捕获长期依赖关系。但是，这些架构需要代表特定过程的大量数据，而这些数据并不总是可用的。为了解决这个问题，迁移学习已成为一种机器学习技术，它使预训练模型能够以最少的额外训练适应新任务。本文演示了一个将迁移学习与 transformer 架构相结合的过程，以便在数据稀缺时启用数据驱动的方法来控制任务，例如系统识别和代理控制建模。在本研究中，来自源系统的大量数据用于训练一个转换器，该转换器对可用数据有限的目标系统的动力学进行建模。本文将仅在目标系统数据上训练的模型与使用迁移学习的模型（包括具有物理信息神经网络 （PINN） 组件的修改后的 transformer 架构）的预测性能进行了比较。结果表明，迁移学习将系统识别的预测准确性提高了 45%，迁移学习和 PINN 架构的预测准确性提高了 74%。在代理控制任务中观察到了类似的准确性改进，使用迁移学习时提高了 44%，使用迁移学习和 PINN 架构时提高了 98%。