We present a general variational framework for the training of freeform nonlinearities in layered computational architectures subject to some slope constraints. The regularization that we add to the traditional training loss penalizes the second-order total variation of each trainable activation. The slope constraints allow us to impose properties such as 1-Lipschitz stability, firm non-expansiveness

Deep learning still has drawbacks regarding trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated with trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described

For an arbitrary given span of high dimensional multivariate Chebyshev polynomials, an approach to construct spatial discretizations is presented, i.e., the construction of a sampling set that allows for the unique reconstruction of each polynomial of this span.

In this paper, we study direct and inverse problems for Dirac systems with complex-valued potentials on p-star-shaped graphs. More precisely, we firstly obtain sharp 2-term asymptotics of the corresponding eigenvalues. We then formulate and address a Horváth-type theorem, specifically, if the potentials on p−1 edges of the p-star-shaped graph are predetermined, we demonstrate that the remaining potential

This paper provides an error estimate for the u-series method of the Coulomb interaction in molecular dynamics simulations. We show that the number of truncated Gaussians M in the u-series and the base of interpolation nodes b in the bilateral serial approximation are two key parameters for the algorithm accuracy, and that the errors converge as O(b−M) for the energy and O(b−3M) for the force. Error

The W-random graphs provide a flexible framework for modeling large random networks. Using the Large Deviation Principle (LDP) for W-random graphs from [19], we prove the LDP for the corresponding class of random symmetric Hilbert-Schmidt integral operators. Our main result describes how the eigenvalues and the eigenspaces of the integral operator are affected by large deviations in the underlying

The identification of the parameters of a neural network from finite samples of input-output pairs is often referred to as the teacher-student model, and this model has represented a popular framework for understanding training and generalization. Even if the problem is NP-complete in the worst case, a rapidly growing literature – after adding suitable distributional assumptions – has established finite

We extend the classical Kadec 14 theorem for systems of exponential functions on an interval to frames and atomic decompositions formed by sampling an orbit of a vector under an isometric group representation.

The frame set conjecture for Hermite functions formulated in [13] states that the Gabor frame set for these generators is the largest possible, that is, the time-frequency shifts of the Hermite functions associated with sampling rates α and modulation rates β that avoid all known obstructions lead to Gabor frames for L2(R). By results in [24,25] and [22], it is known that the conjecture is true for

In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery

Functional linear regression is one of the fundamental and well-studied methods in functional data analysis. In this work, we investigate the functional linear regression model within the context of reproducing kernel Hilbert space by employing general spectral regularization to approximate the slope function with certain smoothness assumptions. We establish optimal convergence rates for estimation

Let G be a finite abelian group. Let f:G→C be a signal (i.e. function). The classical uncertainty principle asserts that the product of the size of the support of f and its Fourier transform fˆ, supp(f) and supp(fˆ) respectively, must satisfy the condition:|supp(f)|⋅|supp(fˆ)|≥|G|.

Without imposing light-tailed noise assumptions, we prove that Tikhonov regularization for Gaussian Empirical Gain Maximization (EGM) in a reproducing kernel Hilbert space is consistent and further establish its fast exponential type convergence rates. In the literature, Gaussian EGM was proposed in various contexts to tackle robust estimation problems and has been applied extensively in a great variety

When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, x↦ReLU(Wx), with a random Gaussian m×n matrix W, in a high-dimensional setting where n,m→∞. Recent work connects this problem to spherical integral geometry giving rise to a conjectured sharp injectivity threshold for α=m/n by studying the expected Euler

The regularity of refinable functions has been analysed in an extensive literature and is well-understood in two cases: 1) univariate 2) multivariate with an isotropic dilation matrix. The general (non-isotropic) case offered a great resistance. It was not before 2019 that the non-isotropic case was done by developing the matrix method. In this paper we make the next step and extend the Littlewood-Paley

In this paper, we present a convergence analysis of the Group Projected Subspace Pursuit (GPSP) algorithm proposed by He et al. [26] (Group Projected subspace pursuit for IDENTification of variable coefficient differential equations (GP-IDENT), Journal of Computational Physics, 494, 112526) and extend its application to general tasks of block sparse signal recovery. Given an observation y and sampling

Multi-scale non-Gaussian time-series having stationary increments appear in a wide range of applications, particularly in finance and physics. We introduce stochastic models that capture intermittency phenomena such as crises or bursts of activity, time reversal asymmetries, and that can be estimated from a single realization of size N. Variations at multiple scales are separated with a wavelet transform

This note considers the multidimensional unstructured sparse recovery problems. Examples include Fourier inversion and sparse deconvolution. The eigenmatrix is a data-driven construction with desired approximate eigenvalues and eigenvectors proposed for the one-dimensional problems. This note extends the eigenmatrix approach to multidimensional problems, providing a rather unified treatment for general

The classical beltway problem entails recovering a set of points from their unordered pairwise distances on the circle. This problem can be viewed as a special case of the crystallographic phase retrieval problem of recovering a sparse signal from its periodic autocorrelation. Based on this interpretation, and motivated by cryo-electron microscopy, we suggest a natural generalization to orthogonal

The moving average of the complex modulus of the analytic wavelet transform provides a robust time-scale representation for signals to small time shifts and deformation. In this work, we derive the Wiener chaos expansion of this representation for stationary Gaussian processes by the Malliavin calculus and combinatorial techniques. The expansion allows us to obtain a lower bound for the Wasserstein

This paper describes a trapezoidal quadrature method for the discretization of weakly singular, and hypersingular boundary integral operators with complex symmetric quadratic forms. Such integral operators naturally arise when complex coordinate methods or complexified contour methods are used for the solution of time-harmonic acoustic and electromagnetic interface problems in three dimensions. The

An equichordal tight fusion frame (▪) is a finite sequence of equi-dimensional subspaces of a Euclidean space that achieves equality in Conway, Hardin and Sloane's simplex bound. Every ▪ is a type of optimal Grassmannian code, being a way to arrange a given number of members of a Grassmannian so that the minimal chordal distance between any pair of them is as large as possible. Any nontrivial ▪ has

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces

Despite the success of Deep Learning (DL) serious reliability issues such as non-robustness persist. An interesting aspect is, whether these problems arise due to insufficient tools or fundamental limitations of DL. We study this question from the computability perspective by characterizing the limits the applied hardware imposes. For this, we focus on the class of inverse problems, which, in particular

Analog-to-digital converters (ADCs) act as a bridge between the analog and digital domains. Two important attributes of any ADC are sampling rate and its dynamic range. For bandlimited signals, the sampling should be above the Nyquist rate. It is also desired that the signals' dynamic range should be within that of the ADC's; otherwise, the signal will be clipped. Nonlinear operators such as modulo

This paper investigates links between the eigenvalues and eigenfunctions of the Laplace-Beltrami operator, and the higher Cheeger constants of smooth Riemannian manifolds, possibly weighted and/or with boundary. The higher Cheeger constants give a loose description of the major geometric features of a manifold. We give a constructive upper bound on the higher Cheeger constants, in terms of the eigenvalue

Spectral estimation is a fundamental task in signal processing. Recent algorithms in quantum phase estimation are concerned with the large noise, large frequency regime of the spectral estimation problem. The recent work in Ding-Epperly-Lin-Zhang proves that the ESPRIT algorithm exhibits superconvergence behavior for the spike locations in terms of the maximum frequency. This note provides a perturbative

Physics-informed neural networks (PINNs) have been demonstrated to be efficient in solving partial differential equations (PDEs) from a variety of experimental perspectives. Some recent studies have also proposed PINN algorithms for PDEs on surfaces, including spheres. However, theoretical understanding of the numerical performance of PINNs, especially PINNs on surfaces or manifolds, is still lacking

We introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as probability measures rather than points in Rn, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available

We investigate the approximation of functions f on a bounded domain Ω⊂Rd by the outputs of single-hidden-layer ReLU neural networks of width n. This form of nonlinear n-term dictionary approximation has been intensely studied since it is the simplest case of neural network approximation (NNA). There are several celebrated approximation results for this form of NNA that introduce novel model classes

We study signals that are sparse in graph spectral domain and develop explicit algorithms to reconstruct the support set as well as partial components from samples on few vertices of the graph. The number of required samples is independent of the total size of the graph and takes only local properties of the graph into account. Our results rely on an operator based framework for subspace methods and

In this paper, we analyze the error estimate of a wavelet frame based image restoration method from degraded and incomplete measurements. We present the error between the underlying original discrete image and the approximate solution which has the minimal ℓ1-norm of the canonical wavelet frame coefficients among all possible solutions. Then we further connect the error estimate for the discrete model

In this paper we formulate Donoho and Logan's large sieve principle for the wavelet transform on the Hardy space, adapting the concept of maximum Nyquist density to the hyperbolic geometry of the underlying space. The results provide deterministic guarantees for L1-minimization methods and hold for a class of mother wavelets that constitutes an orthonormal basis of the Hardy space and can be associated

We develop a data-driven optimal shrinkage algorithm, named extended OptShrink (eOptShrink), for matrix denoising with high-dimensional noise and a separable covariance structure. This noise is colored and dependent across samples. The algorithm leverages the asymptotic behavior of singular values and vectors of the noisy data's random matrix. Our theory includes the sticking property of non-outlier

The generalized translation invariant (GTI) systems unify the discrete frame theory of generalized shift-invariant systems with its continuous version, such as wavelets, shearlets, Gabor transforms, and others. This article provides sufficient conditions to construct pairwise orthogonal Parseval GTI frames in L2(G) satisfying the local integrability condition (LIC) and having the Calderón sum one,

Sparse binary matrices are of great interest in the field of sparse recovery, nonnegative compressed sensing, statistics in networks, and theoretical computer science. This class of matrices makes it possible to perform signal recovery with lower storage costs and faster decoding algorithms. In particular, Bernoulli () matrices formed by independent identically distributed (i.i.d.) Bernoulli () random

The diffusion maps embedding of data lying on a manifold has shown success in tasks such as dimensionality reduction, clustering, and data visualization. In this work, we consider embedding data sets that were sampled from a manifold which is closed under the action of a continuous matrix group. An example of such a data set is images whose planar rotations are arbitrary. The -invariant graph Laplacian

Given a finite group, we study the Gaussian series of the matrices in the image of its left regular representation. We propose such random matrices as a benchmark for improvements to the noncommutative Khintchine inequality, and we highlight an application to the matrix Spencer conjecture.

Spherical needlets were introduced by Narcowich, Petrushev, and Ward to provide a multiresolution sequence of polynomial approximations to functions on the sphere. The needlet construction makes use of integration rules that are exact for polynomials up to a given degree. The aim of the present paper is to relax the exactness of the integration rules by replacing them with QMC designs as introduced

The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing push-forward maps on the measures used to represent functions . An activation function of particular interest is the rectified

In this paper we investigate new results on the theory of superoscillations using time-frequency analysis tools and techniques such as the short-time Fourier transform (STFT) and the Zak transform. We start by studying how the short-time Fourier transform acts on superoscillation sequences. We then apply the supershift property to prove that the short-time Fourier transform preserves the superoscillatory

We show that spectral data of the Koopman operator arising from an analytic expanding circle map can be effectively calculated using an EDMD-type algorithm combining a collocation method of order with a Galerkin method of order . The main result is that if , where is an explicitly given positive number quantifying by how much expands concentric annuli containing the unit circle, then the method converges

In data science, a number of applications have been emerging involving data recovery from permutations. Here, we study this problem theoretically for data organized in a rank-deficient matrix. Specifically, we give unique recovery guarantees for matrices of bounded rank that have undergone arbitrary permutations of their entries. We use methods and results of commutative algebra and algebraic geometry

In this paper we establish accuracy bounds of Prony's method (PM) for recovery of sparse measures from incomplete and noisy frequency measurements, or the so-called problem of super-resolution, when the minimal separation between the points in the support of the measure may be much smaller than the Rayleigh limit. In particular, we show that PM is optimal with respect to the previously established

The problem of verifying the nonnegativity of a function on a finite abelian group is a long-standing challenging problem. The basic representation theory of finite groups indicates that a function on a finite abelian group can be written as a linear combination of characters of irreducible representations of by , where is the dual group of consisting of all characters of and is the of at . In this

After re-casting the wavelet construction problem as a feasibility problem with constraints arising from the requirements of compact support, smoothness and orthogonality, the Douglas–Rachford algorithm is employed in the search for multi-dimensional, non-separable, compactly supported, smooth, orthogonal, multiresolution wavelets in the case of translations along the integer lattice and isotropic

It is usually difficult to study the structure of the spectra for the measures in and higher dimensions. In this paper, by employing the projective techniques and our previous results on the line we prove that the Beurling dimension of spectra for a class of random convolutions in satisfies an intermediate value property.

In this paper, we study the spectral estimation problem of estimating the locations of a fixed number of point sources given multiple snapshots of Fourier measurements in a bounded domain. We aim to provide a mathematical foundation for sparsity-based super-resolution in such spectral estimation problems in both one- and multi-dimensional spaces. In particular, we estimate the resolution and stability

This paper focuses on parameter selection issues of kernel ridge regression (KRR). Due to special spectral properties of KRR, we find that delicate subdivision of the parameter interval shrinks the difference between two successive KRR estimates. Based on this observation, we develop an early-stopping type parameter selection strategy for KRR according to the so-called Lepskii-type principle. Theoretical

Consider the quotient of a Hilbert space by a subgroup of its automorphisms. We study whether this orbit space can be embedded into a Hilbert space by a bilipschitz map, and we identify constraints on such embeddings.

We derive upper bounds on the Wasserstein distance (), with respect to sup-norm, between any continuous valued random field indexed by the -sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. However, an adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models. In this

In this paper, we perform the joint study of scale-invariant operators and self-similar processes of complex order. More precisely, we introduce general families of scale-invariant complex-order fractional-derivation and integration operators by constructing them in the Fourier domain. We analyze these operators in detail, with special emphasis on the decay properties of their output. We further use