In the last decade, conversational artificial intelligence (AI) systems have been widely employed to address people’s real-life needs across various different environments and settings. At the same time, users’ expectations of these systems have been on the rise as they expect more contextual and personalized interactions with continuous learning systems, akin to their expectation in human-human interactions. Modular systems constructed as pipelines of machine learning models and trained through supervised learning paradigms often struggle to improve user experience due to the less-than-ideal, slow accuracy improvements they undergo, and the privacy concerns associated with manual annotation. Inspired by how humans learn from their experiences and interactions, this article proposes a comprehensive self-learning framework designed to tackle these challenges for large-scale conversational AI systems, fostering continuous automated learning. The proposed self-learning framework comprises three elements: feedback collection, feedback interpretation, and learning mechanisms. Without the need for annotators in the loop, a self-learning conversational AI system autonomously uses a feedback interpreter to subscribe to, interpret, and utilize user feedback to adapt its behaviors through various learning mechanisms. First, the elements of the self-learning framework are described and then applied to Alexa, a large-scale conversational AI system. Subsequently, this article presents its effectiveness in reducing user-perceived defects. Finally, it explores the implications of self-learning for general AI systems and suggests future directions.

中文翻译：

---

大规模对话式人工智能系统的自学习框架


在过去的十年中，对话式人工智能（AI）系统已被广泛应用于满足人们在各种不同环境和设置中的现实生活需求。与此同时，用户对这些系统的期望一直在上升，因为他们期望与持续学习系统进行更多上下文和个性化的交互，类似于他们对人与人交互的期望。作为机器学习模型管道构建并通过监督学习范式进行训练的模块化系统通常难以改善用户体验，因为它们所经历的不太理想、准确性改进缓慢，以及与手动注释相关的隐私问题。受人类如何从经验和交互中学习的启发，本文提出了一个全面的自学习框架，旨在应对大规模对话人工智能系统的这些挑战，促进持续的自动化学习。所提出的自学习框架包括三个要素：反馈收集、反馈解释和学习机制。在循环中不需要注释器的情况下，自学习会话人工智能系统可以自主地使用反馈解释器来订阅、解释和利用用户反馈，通过各种学习机制来调整其行为。首先，描述自学习框架的要素，然后将其应用于大型对话式人工智能系统 Alexa。随后，本文介绍了它在减少用户感知的缺陷方面的有效性。最后，它探讨了自学习对通用人工智能系统的影响，并提出了未来的方向。