State-of-the-art microservices have been attracting more attention in recent years. A broad spectrum of online interactive applications are now programmed to service chains on the cloud, seeking better system scalability and lower operating costs. Different from the conventional batch jobs, most of these applications consist of multiple stand-alone services that communicate with each other. These step-by-step operations unavoidably introduce higher latency to the delay-sensitive chained services. In this article, we aim at designing an optimization approach for reducing the latency of chained services. Specifically, presenting the measurement and analysis of chained services on Baidu's cloud platform, our real-world trace indicates that these chained services are suffering from significantly high latency because they are mostly handled by different queues on cloud servers for multiple times. However, such a unique feature introduces significant challenges to optimize a microservice's overall queueing delay. To address this problem, we propose a delay-guaranteed approach to accelerate the overall queueing of chained services while obtaining fairness across all the workloads. Our evaluations on Baidu servers shows that the proposed design can successfully reduce the latency of chained services by 35 percent with minimal impact on other workloads.

中文翻译：

---

快速而公平：基于云的服务链的延迟优化


近年来，最先进的微服务引起了越来越多的关注。现在，广泛的在线交互应用程序被编程为云上的服务链，以寻求更好的系统可扩展性和更低的运营成本。与传统的批处理作业不同，这些应用程序大多数由多个相互通信的独立服务组成。这些分步操作不可避免地会给延迟敏感的链式服务带来更高的延迟。在本文中，我们的目标是设计一种优化方法来减少链式服务的延迟。具体来说，通过对百度云平台上的链式服务的测量和分析，我们的真实世界跟踪表明，这些链式服务由于大多由云服务器上的不同队列多次处理而遭受显着的高延迟。然而，这种独特的功能给优化微服务的整体排队延迟带来了重大挑战。为了解决这个问题，我们提出了一种延迟保证的方法来加速链式服务的整体排队，同时获得所有工作负载的公平性。我们对百度服务器的评估表明，所提出的设计可以成功地将链式服务的延迟降低 35%，同时对其他工作负载的影响最小。