The intersection of machine learning and generalizations of Laver tables seems to have a lot of potential, so your question about this is extremely interesting to me.

Machine learning models from generalized Laver tables?

I have not been able to find any machine learning models that are based on generalized Laver tables that are used for solving problems unrelated to Laver tables. I currently do not directly see any new kinds of deep learning models that one can produce from generalizations of Laver tables, but I would not be surprised if there were a non-standard machine learning model from Laver like algebras that we could use for something like NLP. 

But I have to be skeptical about applying machine learning to generalized Laver tables. Generalizations of Laver tables have intricate structure and complexity but this complexity is ordered. Furthermore, there is a very large supply of Laver-like algebras (even with 2 generators) for potentially any situation. Laver-like algebras are also very easy to compute. While backtracking may potentially take an exponential amount of time, when I have computed Laver-like algebras, the backtracking algorithm seems to always terminate quickly unless it is producing a large quantity or Laver-like algebras or larger Laver-like algebras. But with all this being said, Laver-like algebras seem to lack the steerability that we need to apply these algebras to solving real-world problems. For example, try interpreting (in a model theoretic sense) modular arithmetic modulo 13 in a Laver-like algebra. That is quite hard to do because Laver-like algebras are not built for that sort of thing.

Here are a couple of avenues that I see as most promising for producing new machine learning algorithms from Laver-like algebras and other structures.

Let $X$ be a set, and let $\ast$ be a binary operation on $X$ that satisfies the self-distributivity identity $x\ast (y\ast z)=(x\ast y)\ast (x\ast z)$. Define the right powers $x^{\left[n\right]}$ for $n\ge 1$ inductively by setting $x^{\left[1\right]}=x$ and $x^{[n+1]}=x\ast x^{\left[n\right]}$. We say that $(X,\ast ,1)$ is a reduced nilpotent self-distributive algebra if $1$ satisfies the identities $1\ast x=x,x\ast 1=1$ for $x\in X$ and if for all $x\in X$ there is an $n\ge 1$ with $x^{\left[n\right]}=1$. A reduced Laver-like algebra is a reduced nilpotent self-distributive algebra $(X,\ast )$ where if $x_n\in X$ for each $n\ge 0$, then $x_0\ast \cdots \ast x_N=1$ for some $N$. Here we make the convention to put the implied parentheses on the left so that $a\ast b\ast c\ast d=\left(\right(a\ast b)\ast c)\ast d$.

Reduced nilpotent self-distributive algebras have most of the things that one would expect. Reduced nilpotent self-distributive algebras are often equipped with a composition operation. Reduced nilpotent self-distributive algebras have a notion of a critical point, and if our reduced nilpotent self-distributive algebra is endowed with a composition operation, the set of all critical points in the algebra forms a Heyting algebra. If $(X,\ast )$ is a self-distributive algebra, then we can define $x{\ast }_{0}y=y,x{\ast }_{n+1}y=x{\ast }_n(x\ast y)=x\ast \left(x{\ast }_{n}y\right)$ and$x{\ast }_{\infty }y={lim}_{n\to \infty }x{\ast }_{n}y$ in the discrete topology (this limit always exists for nilpotent self-distributive algebras). We define $x⪯y$ precisely when $x{\ast }_{\infty }y=1$ and $x\simeq y$ precisely when $x⪯y⪯x$. We can define $\text{crit}\left(x\right)$ to be the equivalence class of $x$ with respect to $\simeq$ and $\text{crit}\left[X\right]=X/\simeq$. The operations on $\text{crit}\left[X\right]$ are$\text{crit}\left(x\right)\to \text{crit}\left(y\right)=\text{crit}\left(x{\ast }_{\infty }y\right)$ and $\text{crit}\left(x\right)\wedge \text{crit}\left(y\right)=\text{crit}(x\circ y)$ whenever we have our composition operation $\circ$. One can use computer calculations to add another critical point to a reduced nilpotent self-distributive algebra and obtain a new reduced nilpotent self-distributive algebra with the same number of generators but with another critical point on top (and this new self-distributive algebra will be sub-directly irreducible). Therefore, by taking subdirect products and adding new critical points, we have an endless supply of reduced nilpotent self-distributive algebras with only 2 generators. I also know how to expand reduced nilpotent self-distributive algebras horizontally in the sense that given a finite reduced nilpotent self-distributive algebra $(X,\ast )$ that is not a Laver-like algebra, we can obtain another finite reduced nilpotent self-distributive algebra $(Y,\ast )$ where $X,Y$ are both generated by the same number of elements and these algebras both have the same implication algebra of critical points but $|Y|>>|X|$ but there is a surjective homomorphism $\varphi :Y\to X$. The point is that we have techniques for producing new nilpotent self-distributive algebras from old ones, and going deep into those new nilpotent self-distributive algebras.

Since reduced nilpotent self-distributive algebras are closed under finite subdirect products, subalgebras, and quotients, and since we have techniques for producing many nilpotent self-distributive algebras, perhaps one can make a ML model from these algebraic structures. On the other hand, there seems to be less of a connection between reduced nilpotent self-distributive algebras and large cardinals and non-commutative polynomials than with Laver-like algebras, so perhaps it is sufficient to stick to Laver-like algebras as a platform for constructing ML models.

From Laver-like algebras, we can also produce non-commutative polynomials. If $(X,\ast )$ is a Laver-like algebra, and $(x_a{)}_{a\in A}$ is a generating set for $X$, then for each non-maximal critical point $\alpha \in \text{crit}\left[X\right]$, we can define a non-commutative polynomial $p_{\alpha }\left(\right(x_a{)}_{a\in A})$ to be the sum of all non-commutative monomials of the form $x_{a_1}\dots x_{a_n}$ where $a_1\dots a_n\in A^{\ast }$ and $\text{crit}(a_1\ast \cdots \ast a_n)=\alpha$ but where $\text{crit}(a_1\ast \cdots \ast a_m)<\alpha$ for $1\le m<n$. These non-commutative polynomials capture all the information behind the Laver-like algebras since one can reconstruct the entire Laver-like algebra up to critical equivalence from these non-commutative polynomials. These non-commutative polynomials don't work as well for nilpotent self-distributive algebras; for nilpotent self-distributive algebras, these non-commutative polynomials will instead be rational expressions and one will not be able to recover the entire nilpotent self-distributive algebra from these rational expressions. 

I have used these non-commutative polynomials to construct the infinite product formula

$\dots p_r(x_1,\dots ,x_n)\dots p_0(x_1,\dots ,x_n)=\frac{1}{1-(x_1+\cdots +x_n)}$ where the polynomials $p_n(x_1,\dots ,x_n)$ are obtained from $n$ different non-trivial rank-into-rank embeddings $j_1,\dots ,j_n:V_{\lambda }\to V_{\lambda }$. This means that the non-commutative ring operations $+,\cdot$ in the rings of non-commutative polynomials are meaningful for Laver-like algebras.

If I wanted to interpret and make sense of a Laver-like algebra, I would use these non-commutative polynomials. Since non-commutative polynomials are closely related to strings (for NLP) and since the variables in these non-commutative polynomials can be substituted with matrices, I would not be surprised if one can turn Laver-like algebras into a full ML model using these non-commutative polynomials in order to solve a problem unrelated to Laver-like algebras.

Perhaps, one can also use strong large cardinal hypotheses and logic to produce ML models from Laver-like algebras. Since the existence of a non-trivial rank-into-rank embedding is among the strongest of all large cardinal hypotheses, and since one can produce useful theorems about natural looking finite algebraic structures from these large cardinal hypotheses, perhaps the interplay between the logic using large cardinal hypotheses and finite algebraic structures may be able to produce ML models? For example, one can automate the process of using elementarity to prove the existence of finite algebraic structures. After one has proven the existence of these structures using large cardinal hypotheses, one can do a search using backtracking in order to actually find candidates for these algebraic structures (or if the backtracking algorithm is exhaustive and turns up nothing, then we have a contradiction in the large cardinal hierarchy or a programming error. Mathematicians need to expend a lot of effort into finding an inconsistency in the large cardinal hierarchy this way because I am confident that they won't find such an inconsistency they will instead find plenty of near misses.). We can automate the process of producing examples of Laver-like algebras that satisfy conditions that were first established using large cardinal hypotheses, and we can perhaps completely describe these Laver-like algebras (up-to-critical equivalence) using our exhaustive backtracking process. This approach can be used to produce a lot of data about rank-into-rank embeddings, but do not see a clear path that allows us to apply this data outside of set theory and Laver-like algebras.

Using deep learning to investigate generalized Laver tables

We can certainly apply existing deep learning and machine learning techniques to classical and multigenic Laver tables.

The n-th classical Laver table is the unique self-distributive algebraic structure $A_n=\left(\right\{1,\dots ,2^n\},{\ast }_n)$ where$x{\ast }_{n}1=x+1\mathrm{mod}{2}^n$ whenever $x\in \{1,\dots ,2^n\}$. The classical Laver tables are up-to-isomorphism the only nilpotent self-distributive algebras generated by a single element.

Problem: Compute the $n$-th classical Laver table for as large $n$ as we can achieve.

Solution: Randall Dougherty in his 1995 paper Critical points in an algebra of elementary embeddings II gave an algorithm for computing the 48-th classical Laver table $A_{48}$ and with machine learning, I have personally gotten up to $A_{768}$ (I think I have a couple of errors, but those are not too significant), and I have some of the computation of $A_{1024}$.

Suppose now that $2^N\le n\le 3\cdot 2^N$. Then Dougherty has shown that one can easily recover the entire function ${\ast }_n$ from the restriction $L_{N,n}:\{1,\dots ,2^{2^N}\}\times \{1,\dots ,2^n\}\to \{1,\dots ,2^n\}$ where $L_{N,n}(x,y)=x{\ast }_{n}y$ for all $x,y$. Suppose now that $1\le x\le 2^n$. Then let $o_n\left(x\right)$ denote the least natural number $m$ such that $x{\ast }_n{2}^m=2^n$. Then there is a number ${\theta }_{n+1}\left(x\right)$ called the threshold of $x$ at $A_{n+1}$ such that $0\le {\theta }_{n+1}\left(x\right)\le 2^{o_n\left(x\right)}$ and ${\theta }_{n+1}\left(2^n\right)=0$ and such that for every $y$ with $1\le y\le 2^n$, we have $x{\ast }_{n+1}y=x{\ast }_{n}y$ whenever $1\le y\le {\theta }_{n+1}\left(x\right)$ and$x{\ast }_{n+1}y=\left(x{\ast }_{n}y\right)+2^n$ for ${\theta }_{n+1}\left(x\right)<y\le 2^{o_n\left(x\right)}$. The classical Laver table $A_{n+1}$ can be easily recovered from the table $A_n$ and the threshold function ${\theta }_{n+1}$. To go beyond Dougherty's calculation of $A_{48}$ to $A_{768}$ and beyond, we exploit a couple of observations about the threshold function ${\theta }_{n+1}$:

i: in most cases, ${\theta }_{n+2}\left(x\right)={\theta }_{n+1}\left(x\right)$, and

ii: in most cases, ${\theta }_{n+1}\left(x\right)=2^i-2^j$ for some $i,j$.

Let $E\left(n\right)=\{x\in \{1,\dots ,2^{n-2}\}:{\theta }_{n-1}(x)\ne {\theta }_n(x\left)\right\}$, and let $E^{♯}\left(n\right)=E\left(n\right)\cap \{1,\dots ,2^N-2\}$ where $N$ is the least natural number where $n\le 3\cdot 2^N$ and $T_n=\left\{\right(x,{\theta }_n\left(x\right)):x\in E^{♯}(n\left)\right\}$. Then one can easily compute $A_n$ from $A_{n-1}$ and the data $T_n$ by using Dougherty's algorithm. Now the only thing that we need to do is compute $T_n$ in the first place. It turns out that computing $T_n$ is quite easy except when $n$ is of the form $2^r+1$ or $3\cdot 2^r+1$ for some $r$. In the case, when $n=2^r+1$ or $n=3\cdot 2^r+1$, we initially set $T_n^{\text{temp}}=\varnothing .$ We then repeatedly modify $T_n^{\text{temp}}$ until we can no longer find any contradiction in the statement $T_n=T_n^{\text{temp}}$. Computing $T_n$ essentially amounts to finding new elements in the set $E^{♯}\left(n\right)$ and we find new elements in $E^{♯}\left(n\right)$ using some form of machine learning.

In my calculation of $A_{768}$, I used operations like bitwise AND, OR, and the bitwise majority operation to combine old elements in $E^{♯}\left(n\right)$ to find new elements in $E^{♯}\left(n\right)$, and I also employed other techniques similar to this. I did not use any neural networks though, but using neural networks seems to be a reasonable strategy, but we need to use the right kinds of neural networks.

My idea is to use something similar to a CNN where the linear layers are not as general as possible, but the linear layers are structured in a way that reduces the number of weights and so that the structure of the linear layers is compatible with the structure in the data. The elements in $E^{♯}\left(n\right)$ belong to $\{1,\dots ,2^N\}$ and $\{1,\dots ,2^N\}$ has a natural tensor product structure. One can therefore consider $E^{♯}\left(n\right)$ to be a subset of $(R^2{)}^{\otimes n}$. Furthermore, a tuple of $r$ elements in $E^{♯}\left(n\right)$ can be considered as a subset of $\left(R^2\right){)}^{\otimes n}\otimes R^r$. This means that the linear layers from $\left(R^2\right){)}^{\otimes n}\otimes R^r$ to $\left(R^2\right){)}^{\otimes n}\otimes R^r$ should be Kronecker products $A_1\otimes \cdots \otimes A_{n+1}$ or Kronecker sums $A_1\oplus \cdots \oplus A_{n+1}$ (it seems like Kronecker sums with residual layers would work better than Kronecker products). Recall that the Kronecker product and Kronecker sum of matrices $A,B$ are defined by setting $(A\otimes B)(u\otimes v)=Au\otimes Bv$ and $A\oplus B=(A\otimes 1)+(1\otimes B)$ respectively. Perhaps neural networks can be used to generate new elements in $E^{♯}\left(n\right)$. These generative neural networks do not have to be too large nor do they have to be too accurate. A neural network that is 50 percent accurate will be better than a neural network that is 90 percent accurate but takes 10 times as long to compute and is much harder to retrain once most of the elements in $E^{♯}\left(n\right)$ have already been obtained and when the neural network has trouble finding new elements. I also favor smaller neural networks because one can compute $A_{768}$ without using complicated techniques in the first place. I still need to figure out the rest of the details for the generative neural network that finds new elements of $E^{♯}\left(n\right)$ since I want it to be simple and easy to compute so that it is competitive with things like bitwise operations.

Problem: Translate Laver-like algebras into data (like sequences of vectors or matrices) into a form that machine learning models can work with.

Solution: The non-commutative polynomials $p_0,\dots ,p_n$ are already in a form that is easier for machine learning algorithms to use. I was able to apply an algorithm that I originally developed for analyzing block ciphers in order to turn the non-commutative polynomials $p_0,\dots ,p_n$ into unique sequences of real matrices $A_0,\dots ,A_n$ (these real matrices do not seem to depend on the initialization since the gradient ascent always seems to converge to the same local maximum). One can then feed this sequence of real matrices $A_0,\dots ,A_n$ into a transformer to solve whatever problem one wants to solve about Laver-like algebras. One can also represent a Laver-like algebra as a 2-dimensional image and then pass that image through a CNN to learn about the Laver-like algebra. This technique may only be used for Laver-like algebras that are small enough for CNNs to fully see, so I do not recommend CNNs for Laver-like algebras, but this is at least a proof-of-concept.

Problem: Estimate the number of small enough Laver-like algebras (up-to-critical equivalence) that satisfy certain properties.

Solution: The set of all multigenic Laver tables over an alphabet $A$ forms a rooted tree. We can therefore apply the technique for estimating the number of nodes in a rooted tree described in the 1975 paper Estimating the Efficiency of Backtrack Programs by Donald Knuth. This estimator is unbiased (the mean of the estimated value is actually the number of nodes in the tree), but it will have a very high variance for estimating the number of multigenic Laver tables. In order to reduce the variance for this estimator, we need to assign better probabilities when finding a random path from the root to the leaves, and we should be able to use transformers to assign these probabilities.

I am pretty sure that there are plenty of other ways to use machine learning to investigate Laver-like algebras.

Now, nilpotent self-distributive algebras may be applicable in cryptography (they seem like suitable platforms for the Kalka-Teicher key exchange but nobody has researched this). Perhaps a transformer that learns about Laver-like algebras would do better on some unrelated task than a transformer that was never trained using Laver-like algebras. But I do not see any other practical applications of nilpotent Laver-like algebras at the moment. This means that Laver-like algebras are currently a relatively safe problem for investigating machine learning algorithms, so Laver-like algebras are perhaps applicable to AI safety since I currently do not see any way of misaligning an AI for solving a problem related to Laver-like algebras.

This post gives an example of some calculations that I did using my own machine learning algorithm. These calculations work out nicely which indicates that the machine learning algorithm I am using is interpretable (and it is much more interpretable than any neural network would be). These calculations show that one can begin with old mathematical structures and produce new mathematical structures, and it seems feasible to completely automate this process to continue to produce more mathematical structures. The machine learning models that I use are linear, but it seems like we can get highly non-trivial results simply by iterating the procedure of obtaining new structures from old using machine learning.

I made a similar post to this one about 7 months ago, but I decided to revisit this experiment with more general algorithms and I have obtained experimental results which I think look nice.

To illustrate how this works, we start off with the octonions. The octonions consists of an 8-dimensional inner product space $V$ together with a bilinear operation $\ast$ and a unit $1\in V$ where $1\ast v=v\ast 1=v$ for all $v\in V$ and where $\parallel u\ast v\parallel =\parallel u\parallel \cdot \parallel v\parallel$ for all $u,v\in V$. The octonions are uniquely determined up to isomorphism from these properties. The operation $\ast$ is non-associative, but the $\ast$ is closely related to the quaternions and complex numbers. If we take a single element in $v\in V\setminus \text{Span}\left(1\right)$, then $\{1,v\}$ generates a subalgebra of $(V,\ast )$ isomorphic to the field of complex numbers, and if $u,v\in V$ and $\{1,u,v\}$ are linearly independent, then $\{1,u,v,u\ast v\}$ spans a subalgebra of $V$ isomorphic to the division ring of quaternions. For this reason, one commonly thinks of the octonions as the best way to extend the division ring of quaternions to a larger algebraic structure in the same way that the quaternions extend the field of complex numbers. But since the octonions are non-associative, they cannot be used to construct matrices, so they are not as well-known as the quaternions (and the construction of the octonions is more complicated too)

Suppose now that $\{e_0,e_1,\dots ,e_7\}$ is an orthonormal basis for the division ring of octonions with $e_0=1$. Then define matrices $A_0,\dots ,A_7:V\to V$ by setting $A_{j}v=e_j\ast v$ for all $j$. Our goal is to transform $(A_0,\dots ,A_7)$ into other tuples of matrices that satisfy similar properties.

If $(A_1,\dots ,A_r),(B_1,\dots ,B_r)$ are matrices, then define the $L_2$

-spectral radius similarity between $(A_1,\dots ,A_r)$ and $(B_1,\dots ,B_r)$ as

$\parallel (A_1,\dots ,A_r)\simeq (B_1,\dots ,B_r){\parallel }_2=$

$\frac{\rho (A_1\otimes \overline{B_1}+\cdots +A_r\otimes \overline{B_r})}{\rho (A_1\otimes \overline{A_1}+\cdots +A_r\otimes \overline{A_r}{)}^{1/2}\cdot \rho (B_1\otimes \overline{B_1}+\cdots +B_r\otimes \overline{B_r}{)}^{1/2}}$

where $\rho$ denotes the spectral radius, $\otimes$ is the tensor product, and $\overline{X}$ is the complex conjugate of $X$ applied elementwise.

Let $d\in \{1,\dots ,8\}$, and let $F_d,G_d,H_d$ denote the maximum value of the fitness level $8\cdot \parallel (A_0,\dots ,A_7)\simeq (X_0,\dots ,X_7){\parallel }^2$ such that each $X_j$ is a complex $d\times d$ anti-symmetric matrix ($X=-X^T$), a complex $d\times d$ symmetric matrix ($X=X^T$), and a complex $d\times d$-Hermitian matrix ($X=X^{\ast }$) respectively.

The following calculations were obtained through gradient ascent, so I have no mathematical proof that the values obtained are actually correct.

                                    $G_1=2$,                                  $H_1=1$

                                    $G_2=3$,                                 $H_2=3$

$F_3=1+\sqrt{3}$ ,          $G_3=3.5$,                              $H_3=1+2\sqrt{2}$

$F_4=4$ ,                      $G_4=4$,                                 $H_4=1+3\sqrt{2}$

$F_5=(5+\sqrt{13})/2$, $G_5=4.5$,                              $H_5\approx 5.27155841$

$F_6=5$ ,                      $G_6=5$,                                 $H_6=3+2\sqrt{2}$

$F_7=6$ ,                      $G_7=2+2\sqrt{3}\approx 5.4641$, $H_7=1+2\sqrt{7}$

$F_8=7$ ,                      $G_8=6$,                                 $H_8=7$

Observe that with at most one exception, all of these values $F_d,G_d,H_d$ are algebraic half integers. This indicates that the fitness function that we maximize to produce $F_d,G_d,H_d$ behaves mathematically and can be used to produce new tuples $(X_1,\dots ,X_r)$ from old ones $(A_1,\dots ,A_r)$. Furthermore, an AI can determine whether something notable is going on with the new tuple $(X_1,\dots ,X_r)$ in several ways. For example, if $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }^2$ has low algebraic degree at the local maximum, then $(X_1,\dots ,X_r)$ is likely notable and likely behaves mathematically (and is probably quite interpretable too).

The good behavior of $F_d,G_d,H_d$ demonstrates that the octonions are compatible with the $L_2$-spectral radius similarity. The operators $(A_0,\dots ,A_7)$ are all orthogonal, and one can take the tuple $(A_0,\dots ,A_7)$ as a mixed unitary quantum channel that is very similar to the completely depolarizing channel. The completely depolarizing channel completely mixes every quantum state while the mixture of orthogonal mappings $(A_0,\dots ,A_7)$ completely mixes every real state. The $L_2$-spectral radius similarity works very well with the completely depolarizing channel, so one should expect for the $L_2$-spectral radius similarity to also behave well with the octonions.

It is time for us to interpret some linear machine learning models that I have been working on. These models are linear, but I can generalize these algorithms to produce multilinear models which have stronger capabilities while still behaving mathematically. Since one can stack the layers to make non-linear models, these types of machine learning algorithms seem to have enough performance to be more relevant for AI safety.

Our goal is to transform a list of $n\times n$-matrices $(A_1,...,A_r)$ into a new and simplified list of $d\times d$-matrices $(X_1,\dots ,X_r)$. There are several ways in which we would like to simplify the matrices. For example, we would sometimes simply like for $d<n$, but in other cases, we would like the matrices $X_j$ to all be real symmetric, complex symmetric, real Hermitian, complex Hermitian, complex anti-symmetric, etc. 

We measure similarity between tuples of matrices using spectral radii. Suppose that $(A_1,\dots ,A_r)$ are $n\times n$-matrices and $(X_1,\dots ,X_r)$ are $d\times d$-matrices. Then define an operator $\Gamma (A_1,\dots ,A_r:X_1,\dots ,X_r)$ mapping $n\times d$ matrices to $n\times d$

-matrices by setting $\Gamma (A_1,\dots ,A_r:X_1,\dots ,X_r)\left(X\right)=A_{1}X{X}_1^{\ast }+\dots A_{r}X{X}_r^{\ast }$. Then define $\Phi (X_1,\dots ,X_r)=\Gamma (X_1,\dots ,X_r;X_1,\dots ,X_r)$. Define the similarity between $(A_1,\dots ,A_r)$ and $(X_1,\dots ,X_r)$ by setting

$\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$

$=\frac{\rho \left(\Gamma \right(A_1,\dots ,A_r;X_1,\dots ,X_r\left)\right)}{\rho \left(\Phi \right(A_1,\dots ,A_r\left){)}^{1/2}\rho \right(\Phi (X_1,\dots ,X_r){)}^{1/2}}$

where $\rho$ denotes the spectral radius. Here, $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$ should be thought of as a generalization of the cosine similarity to tuples of matrices. And $\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$ is always a real number in $[0,1]$, so this is a sensible notion of similarity.

Suppose that $K$ is either the field of real or complex numbers. Let $M_n\left(K\right)$ denote the set of $n$ by $n$ matrices over $K$. 

Let $n,d$ be positive integers. Let $T:M_d\left(K\right)\to M_d\left(K\right)$ denote a projection operator. Here, $T$ is a real-linear operator, but if $K$ is not complex, then $T$ is not necessarily complex linear. Here are a few examples of such linear operators $T$ that work:

 $K=C:T_1\left(X\right)=(X+X^T)/2$ (Complex symmetric)

$K=C:T_2\left(X\right)=(X-X^T)/2$ (Complex anti-symmetric)

$K=C:T_3\left(X\right)=(X+X^{\ast })/2$ (Complex Hermitian)

$K=C:T_4\left(X\right)=\text{Re}\left(X\right)$ (real, the real part taken elementwise).

$K=R:T_5\left(X\right)=(X+X^T)/2$ (Real symmetric)

$K=R:T_6\left(X\right)=(X-X^T)/2$ (Real anti-symmetric)

$K=C:T_7\left(X\right)=\text{Re}\left(X\right)+\text{Re}(X{)}^T$ (real symmetric)

$K=C:T_8\left(X\right)=\text{Re}\left(X\right)-\text{Re}(X{)}^T$ (real anti-symmetric)

Caution: These are special projection operators on spaces of matrices. The following algorithms do not behave well for general projection operators; they mainly behave well for $T_1,\dots ,T_8$ along with operators that I have forgotten about.

We are now ready to describe our machine learning algorithm's input and objective.

Input: $r$-matrices $A_1,\dots ,A_r\in M_n\left(K\right)$

Objective: Our goal is to obtain matrices $(X_1,\dots ,X_r)\in M_d\left(K\right)$  where $T\left(X_j\right)=X_j$ for all $j$ but which locally maximizes the similarity$\parallel (A_1,\dots ,A_r)\simeq (X_1,\dots ,X_r){\parallel }_2$.

In this case, we shall call $(X_1,\dots ,X_r)$ an $L_{2,d}$-spectral radius dimensionality reduction (LSRDR) along the subspace $\text{im}\left(T\right).$

LSRDRs along subspaces often perform tricks and are very well-behaved.

If $(X_1,\dots ,X_r),(Y_1,\dots ,Y_r)$ are LSRDRs along subspaces, then there are typically some $\lambda ,C$ where $Y_j=\lambda C{X}_j{C}^{-1}$ for all $j$. Furthermore, if $(X_1,\dots ,X_r)$ is an LSRDR along a subspace, then we can typically find some matrices $R,S$ where$X_j=T\left(R{A}_{j}S\right)$ for all $j$. 

The model $(X_1,\dots ,X_r)$ simplifies since it is encoded into the matrices $R,S$, but this also means that the model $(X_1,\dots ,X_r)$ is a linear model. I have just made these observations about the LSRDRs along subspaces, but they seem to behave mathematically enough for me especially since the matrices $R,S$ tend to have mathematical properties that I can't explain and am still exploring.

Universities are altogether unprofessional, so it is probably best for everyone to shame them and regard the degrees from these universities are completely worthless. Universities promote violence and they refuse to apologize or acknowledge that there is any problem whatsoever.

Since AI interpretability is a big issue for AI safety, let's completely interpret the results of evolutionary computation. 

Disclaimer: This interpretation of the results of AI does not generalize to interpreting deep neural networks. This is a result for interpreting a solution to a very specific problem that is far less complicated than deep learning, and by interpreting, I mean that we iterate a mathematical operation hundreds of times to get an object that is simpler than our original object, so don't get your hopes up too much.

A basis matroid is a pair $(X,M)$ where $X$ is a finite set, and $M\subseteq P\left(X\right)$ where $M$ denotes the power set of $X$ that satisfies the following two properties:

1. If $A,B\in M$, then $|A|=|B|$.
2. if $A,B\in M,A\ne B,a\in A\setminus B$, then there is some $b\in B\setminus A$ with $(A\setminus \{a\left\}\right)\cup \left\{b\right\}\in M$ (the basis exchange property).

I ran a computer experiment where I obtained a matroid $(X,M)$ where $|X|=9$   $|M|=68$ and where each element in $M$ has size $4$ through evolutionary computation, but the population size was kept so low that this evolutionary computation mimicked hill climbing algorithms. Now we need to interpret the matroid $(X,M)$.

The notion of a matroid has many dualities. Our strategy is to apply one of these dualities to the matroid $(X,M)$ so that the dual object is much smaller than the original object $(X,M)$. One may formulate the notion of a matroid in terms of closure systems (flats),hyperplanes, closure operators, lattices, a rank function, independent sets, bases, and circuits. If these seems to complicated, many of these dualities are special cases of other dualities common with ordered sets. For example, the duality between closure systems, closure operators, and ordered sets applies to contexts unrelated to matroids such as in general and point-free topology. And the duality between the basis, circuit, and the hyperplanes may be characterized in terms of rowmotion.

If $(Z,\le )$ is a partially ordered set, then a subset $A\subseteq Z$ is said to be an antichain if whenever $a,b\in A,a\le b$, then $a=b$. In other words, an antichain is a subset $A$ of $Z$ where the restriction of $\le$ to $A$ is equality. We say that a aubset $L$ of $Z$ is downwards closed if whenever $x\le y$ and $y\in L$, then $x\in L$ as well. If $A\subseteq Z$, then let $L\left(A\right)$ denote the smallest downwards closed subset of $Z$ containing $A$. Suppose that $Z$ is a finite poset. If $A$ is an antichain in $Z$, then let $A^{\prime }$ denote the set of all minimal elements in $Z\setminus L\left(A\right)$. Then $A^{\prime }$ is an antichain as well, and the mapping $A\mapsto A^{\prime }$ is a bijection from the set of all antichains in $Z$ to itself. This means that if $A$ is an antichain, then we may define $A^{\left(n\right)}$ for all integers $n$ by setting $A^{\left(0\right)}=A,A^{(n+1)}=(A^{\left(n\right)}{)}^{\prime }$.

If $(X,M)$ is a basis matroid, then $M$ is an antichain in $P\left(X\right)$, so we may apply rowmotion, so we say that $(X,M^{\left(n\right)})$ is an $n$-matroid. In this case, the $1$

-matroids are the circuit matroids while the $-1$-matroids are the hyperplane matroids. Unfortunately, the $n$-matroids have not been characterized for $|n|>1$. We say that the rowmotion order of $(X,M)$ is the least positive integer $n$ where $M^{\left(n\right)}=M$. If $(X,M)$ is a matroid of order $n$, then my computer experiments indicate that $gcd(|X|+2,n)>1$ whichs lends support to the idea that the rowmotion of a matroid is a sensible mathematical notion that may be satisfied mathematically. The notion of rowmotion of a matroid also appears to be a sensible mathematical notion for other reasons; if we apply iteratively apply a bijective operation $g$ (such as a reversible cellular automaton) to a finite object $x$, then that bijective operation will often increase the entropy in the sense that if $x$ has low entropy, then $g^{\left(n\right)}\left(x\right)$ will typically have a high amount of entropy and look like noise. But this is not the case with matroids as $n$-matroids do not appear substantially more complicated than basis matroids. Until and if there is a mundane explanation for this behavior of the rowmotion of matroids, I must consider the notion of rowmotion of matroids to be a mathematically interesting notion even though it is currently not understood by anyone.

With the matroid $(X,M)$ obtained from evolutionary computation, I found that $(X,M)$ has order $1958$ which factorizes as $1958=2\cdot 79\cdot 11$. Set $X=\{1,\dots ,9\}$. By applying rowmotion to this matroid, I found that $M^{\left(342\right)}$={{1, 8, 9},{2, 3, 6, 8},{2, 3, 7, 9},{4, 5},{4, 6, 9},{4, 7, 8},{5, 6, 9},{5, 7, 8}}. If $(X,M^{\left(m\right)})$ is a basis matroid, then $M^{\left(m\right)}=M$, so the set $M^{\left(342\right)}$ is associated with a unique basis matroid. This is the smallest way to represent $(X,M)$ in terms of rowmotion since if $|M^{\left(n\right)}|\le 8$, then $M^{\left(n\right)}=M^{\left(342\right)}$.

I consider this a somewhat satisfactory interpretation of the matroid $(X,M)$ that I have obtained through evolutionary computation, but there is still work to do because nobody has researched the rowmotion operation on matroids and because it would be better to simplify a matroid without needing to go through hundreds of layers of rowmotion. And even if we were to understand matroid rowmotion better, this would not help us too much with AI safety since here this interpretability of the result of evolutionary computation does not generalize to other AI's and it certainly does not apply to deep neural networks.

I made a video here where one may see the rowmotion of this matroid and that video is only slightly interpretable.

 Deep matroid duality visualization: Rowmotion of a matroid 

It turns out that evolutionary computation is not even necessary to construct matroids since Donald Knuth has produced an algorithm that can be used to construct an arbitrary matroid in his 1975 paper on random matroids. But I applied the rowmotion to the matroid in his paper and the resulting 10835-matroid $B^{\left(10835\right)}$={{1, 2, 4, 5},{1, 2, 6, 10},{1, 3, 4, 6},{1, 3, 4, 7, 9},{1, 3, 6, 7, 9},{1, 4, 6, 7},{1, 4, 6, 9},{1, 4, 8, 10},{2, 3, 4, 5, 6, 7, 8, 9, 10}}. It looks like the rowmotion operation is good for simplifying matroids in general. We can uniquely recover the basis matroid from the 10835 matroid since $B^{\left(m\right)}$ is not a basis matroid whenever $0<m\le 10835$.

I have originally developed a machine learning notion which I call an LSRDR ($L_{2,d}$

-spectral radius dimensionality reduction), and LSRDRs (and similar machine learning models) behave mathematically and they have a high level of interpretability which should be good for AI safety. Here, I am giving an example of how LSRDRs behave mathematically and how one can get the most out of interpreting an LSRDR.

Suppose that $n$ is a natural number. Let $N$ denote the quantum channel that takes an $n$ qubit quantum state and selects one of those qubits at random and send that qubit through the completely depolarizing channel (the completely depolarizing channel takes a state as input and returns the completely mixed state as an output).

If $A_1,\dots ,A_r,B_1,\dots ,B_r$ are complex matrices, then define superoperators $\Phi (A_1,\dots ,A_r)$ and $\Gamma (A_1,\dots ,A_r;B_1,\dots ,B_r)$ by setting

$\Phi (A_1,\dots ,A_r)\left(X\right)={\sum }_{k=1}^r{A}_{k}X{A}_k^{\ast }$ and $\Gamma (A_1,\dots ,A_r;B_1,\dots ,B_r)={\sum }_{k=1}^r{A}_{k}X{B}_k^{\ast }$ for all $X$. 

Given tuples of matrices $(A_1,\dots ,A_r),(B_1,\dots ,B_r)$, define the L_2-spectral radius similarity between these tuples of matrices by setting

$\Vert (A_1,\dots ,A_r)\simeq (B_1,\dots ,B_r){\parallel }_2$

$=\frac{\rho \left(\Gamma \right(A_1,\dots ,A_r;B_1,\dots ,B_r\left)\right)}{\rho \left(\Phi \right(A_1,\dots ,A_r\left){)}^{1/2}\rho \right(\Phi (B_1,\dots ,B_r){)}^{1/2}}$.

Suppose now that $A_1,\dots ,A_{4n}$ are matrices where $N=\Phi (A_1,\dots ,A_{4n})$. Let $1\le d\le n$. We say that a tuple of complex $d$ by $d$ matrices $(X_1,\dots ,X_{4n})$ is an LSRDR of $A_1,\dots ,A_{4n}$ if the quantity $\parallel (A_1,\dots ,A_{4n})\simeq (X_1,\dots ,X_{4n}){\parallel }_2$ is locally maximized.

Suppose now that $X_1,\dots ,X_{4n}$ are complex $2\times 2$-matrices and $(X_1,\dots ,X_{4n})$ is an LSRDR of $A_1,\dots ,A_{4n}$. Then my computer experiments indicate that there will be some constant $\lambda$ where $\lambda \Gamma (A_1,\dots ,A_{4n};X_1,\dots ,X_{4n})$ is similar to a positive semidefinite operator with eigenvalues $\{0,\dots ,n+1\}$ and where the eigenvalue $j$ has multiplicity $3\cdot C(n-1,k)+C(n-1,k-2)$ where $C(\cdot ,\cdot )$ denotes the binomial coefficient. I have not had a chance to try to mathematically prove this. Hooray. We have interpreted the LSRDR $(X_1,\dots ,X_{4n})$ of $(A_1,\dots ,A_{4n})$, and I have plenty of other examples of interpreted LSRDRs.

We also have a similar pattern for the spectrum of $\Phi (A_1,\dots ,A_{4n})$. My computer experiments indicate that there is some constant $\lambda$ where $\lambda \cdot \Phi (A_1,\dots ,A_{4n})$ has spectrum $\{0,\dots ,n\}$ where the eigenvalue $j$ has multiplicity $3^{n-j}\cdot C(n,j)$.

In this note, I will continue to demonstrate not only the ways in which LSRDRs ($L_{2,d}$-spectral radius dimensionality reduction) are mathematical but also how one can get the most out of LSRDRs. LSRDRs are one of the types of machine learning that I have been working on, and LSRDRs have characteristics that tell us that LSRDRs are often inherently interpretable which should be good for AI safety.

Suppose that $N$ is the quantum channel that maps a $n$ qubit state to a $n$ qubit state where we select one of the 6 qubits at random and send it through the completely depolarizing channel (the completely depolarizing channel takes a state as an input and returns the completely mixed state as an output). Suppose that $A_1,\dots ,A_{4n}$ are $2^n$ by $2^n$ matrices where $N$ has the Kraus representation $N\left(X\right)={\sum }_{k=1}^{4n}{A}_{k}X{A}_k^{\ast }$. 

The objective is to locally maximize the fitness level $\rho \left({\sum }_{k=1}^{4n}{z}_k{A}_k\right)/\parallel (z_1,\dots ,z_{4n})\parallel$ where the norm in question is the Euclidean norm and where $\rho$ denotes the spectral radius. This is a 1 dimensional case of an LSRDR of the channel $N$.

Let $A={\sum }_{k=1}^{4n}{z}_k{A}_k$ when $(z_1,\dots ,z_{4n})$ is selected to locally maximize the fitness level. Then my empirical calculations show that there is some $\lambda$ where $\lambda {\sum }_{k=1}^{4n}{z}_k{A}_k$is positive semidefinite with eigenvalues $\{0,\dots ,n\}$ and where the eigenvalue $k$ has multiplicity $(\frac{n}{k})$ which is the binomial coefficient. But these are empirical calculations for select values $\lambda$; I have not been able to mathematically prove that this is always the case for all local maxima for the fitness level (I have not tried to come up with a proof).

Here, we have obtained a complete characterization of $A$ up-to-unitary equivalence due to the spectral theorem, so we are quite close to completely interpreting the local maximum for our fitness function. 

I made a few YouTube videos showcasing the process of maximizing the fitness level here.

Spectra of 1 dimensional LSRDRs of 6 qubit noise channel during training

Spectra of 1 dimensional LSRDRs of 7 qubit noise channel during training

Spectra of 1 dimensional LSRDRs of 8 qubit noise channel during training

I will make another post soon about more LSRDRs of a higher dimension of the same channel $N$.