Introduction
This was originally posted here.
I've been researching, for quite some time, the prospect of machine learning on a wider variety of data types than normally considered; things other than tables of numbers and categories. In particular, I want to do ML for program and proof synthesis which requires, at the very least, learning the structures of trees or graphs which don't come from a differentiable domain. Normal ML algorithms can't handle these; though some recent methods, such as graph neural networks and transformers, can be adapted to this domain with some promising results. However, these methods still rely on differentiation. Is this really required? Are we forever doomed to map all our... (read 12235 more words →)
I do think that something could be done. I'm not very familiar with Ant Colony Optimization in particular, and it's not immediately obvious to me how it would be applied here. Specifically, how would it take advantage of the algorithmic complexity if that's all we know beforehand? Often, we're working with infinite parameter spaces where even reasonable subsets will grow exponentially large as we expand them. Is there something like a lazy version of Ant Colony Optimization which can be applied here?
I do think some kind of randomized, learning search would be helpful. I mentioned genetic programming and I think stochastic supercompilation like searches (a la Stoke) would likely work as well.