Joseph Van Name

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

Here are some empirical observations that I have made on August 14, 2023 to August 19, 2023 that are characteristics of the interpretability of my own matrix dimensionality reduction algorithm. These phenomena that we observe do not occur on all inputs (they sometimes occur partially); and it would be nice if there were a more complete mathematical theory with proofs that explains these empirical phenomena. Given a (possibly directed and possibly weighed) graph with n nodes represented as a collection of n×n-matrices A1,…,Ar, we will observe that a dimensionality reduction (X1,…,Xr) of (A1,…,Ar) where each Xi is a d×d-matrix (I call this dimensionality reduction an LSRDR) is in many cases the optimal solution to a combinatorial problem for the graph. In this case, we have a complete interpretation of what the dimensionality reduction algorithm is doing. For this post, let K denote either the field of real numbers or the field of complex numbers (everything also works well when K is the division ring of quaternions). Notation: ρ(A)=limn→∞∥An∥1/n=max{|λ|:λis an eigenvalue ofA} is the spectral radius of the matrix A. AT denotes the transpose of A while A∗ denotes the adjoint of A. We say that a tuples of matrices (A1,…,Ar) is jointly similar to (B1,…,Br) if there is an invertible C with Bj=CAjC−1 for 1≤j≤r. A⊗B denotes the tensor product of A with B. Let 1≤d<n. Suppose that A1,…,Ar are n×n-matrices with entries in K. Then we say that a collection (X1,…,Xr) of d×d-matrices with entries in K is an L2,d-spectral radius dimensionality reduction (abbreviated LSRDR) of A1,…,Ar if the following quantity is locally maximized:ρ(A1⊗(X∗1)T+⋯+Ar⊗(X∗r)T)ρ(X1⊗(X∗1)T+⋯+Xr⊗(X∗r)T)1/2. LSRDRs may be computed using a variation of gradient ascent. If (X1,…,Xr) is an LSRDR of (A1,…,Ar), then one will typically be able to find matrices R,S,P and some λ∈K where Xj=RAjS for 1≤j≤r and where SR=λPand P2=P. We shall call P a L2,d-SRDR projection operator of (A1,…,Ar). (X1⊕0n−d,…,Xr⊕0n−d)

16Aug 19, 2023

Joseph Van Name

Message

116

Approximating arbitrary complex-valued continuous functions

In machine learning, one often wants to uniformly approximate an arbitrary continuous function arbitrarily well using polynomials, neural networks, or something else. But in the case of complex-valued functions, this is more difficult. For example, the limit of holomorphic functions in the topology of uniform convergence on compact sets is...

Nov 17, 20255

Using complex polynomials to approximate arbitrary continuous functions

Machine learning algorithms such as neural networks are supposed to have some sort of universal uniform approximation theorem that shows that they can (at least in principle) learn any possible data set without simply overfitting to the training data. The standard universal approximation theorem applies to shallow neural networks with...

Oct 11, 20255

Spectral radii dimensionality reduction computed without gradient calculations

In this post, I shall describe a fitness function that can be locally maximized without gradient computations. This fitness function is my own. I initially developed this fitness function in order to evaluate block ciphers for cryptocurrency technologies, but I later found that this fitness function may be used to...

May 28, 20255

Joseph Van Name's Shortform

Oct 29, 20232

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

In this post, I shall first describe a new word embedding algorithm that I came up with called a matrix product optimized (MPO) word embedding, and I will prove a theorem that completely interprets this word embedding in the simplest case. While it is probably infeasible to mathematically characterize a...

Sep 4, 20233

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

Aug 19, 202316

When performing a dimensionality reduction on tensors, the trace is often zero.

In this post, we shall define my new dimensionality reduction for tensors in V⊗n where n≥3, and we shall make an empirical observation about the structure of the dimensionality reduction. There are various simple ways of adapting this dimensionality reduction algorithm to tensors in V1⊗⋯⊗Vn and even mixed quantum states...

Aug 2, 20237

LESSWRONG
LW

LESSWRONG
LW

Joseph Van Name

Joseph Van Name

Joseph Van Name

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Spectral radii dimensionality reduction computed without gradient calculations

Approximating arbitrary complex-valued continuous functions

Joseph Van Name

Approximating arbitrary complex-valued continuous functions

Using complex polynomials to approximate arbitrary continuous functions

Spectral radii dimensionality reduction computed without gradient calculations

Joseph Van Name's Shortform

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.

Spectral radii dimensionality reduction computed without gradient calculations

Approximating arbitrary complex-valued continuous functions

Approximating arbitrary complex-valued continuous functions

Using complex polynomials to approximate arbitrary continuous functions

Spectral radii dimensionality reduction computed without gradient calculations

Joseph Van Name's Shortform

Interpreting a matrix-valued word embedding with a mathematically proven characterization of all optima

Interpreting a dimensionality reduction of a collection of matrices as two positive semidefinite block diagonal matrices

When performing a dimensionality reduction on tensors, the trace is often zero.