We consider extensions of the Shannon relative entropy, referred to as f-divergences. Three classical related computational problems are typically associated with these divergences: (a) estimation from moments, (b) computing normalizing integrals, and (c) variational inference in probabilistic models. These problems are related to one another through convex duality, and for all of them, there are many applications throughout data science, and we aim for computationally tractable approximation algorithms that preserve properties of the original problem such as potential convexity or monotonicity. In order to achieve this, we derive a sequence of convex relaxations for computing these divergences from non-centered covariance matrices associated with a given feature vector: starting from the typically non-tractable optimal lower-bound, we consider an additional relaxation based on “sums-of-squares”, which is is now computable in polynomial time as a semidefinite program. We also provide computationally more efficient relaxations based on spectral information divergences from quantum information theory. For all of the tasks above, beyond proposing new relaxations, we derive tractable convex optimization algorithms, and we present illustrations on multivariate trigonometric polynomials and functions on the Boolean hypercube.

我们考虑香农相对熵的扩展，称为f散度。三个经典相关计算问题通常与这些分歧相关：（a）矩估计，（b）计算归一化积分，以及（c）概率模型中的变分推理。这些问题通过凸对偶性相互关联，对于所有这些问题，在整个数据科学中都有许多应用，我们的目标是找到计算上易于处理的近似算法，以保留原始问题的属性，例如潜在的凸性或单调性。为了实现这一点，我们推导出一系列凸松弛，用于从与给定特征向量关联的非中心协方差矩阵计算这些散度：从典型的不可处理的最佳下界开始，我们考虑基于“的附加松弛”平方和”，现在可以作为半定程序在多项式时间内计算。我们还基于量子信息理论的光谱信息散度提供计算上更有效的松弛。对于上述所有任务，除了提出新的松弛之外，我们还推导了易于处理的凸优化算法，并展示了多元三角多项式和布尔超立方体上的函数的插图。