Creating Friendly AI seems to require us humans to either solve most of the outstanding problems in philosophy, or to solve meta-philosophy (i.e., what is the nature of philosophy, how do we practice it, and how should we program an AI to do it?), and to do that in an amount of time measured in decades. I'm not optimistic about our chances of success, but out of these two approaches, the latter seems slightly easier, or at least less effort has already been spent on it. This post tries to take a small step in that direction, by asking a few questions that I think are worth investigating or keeping in the back of our minds, and generally raising awareness and interest in the topic.
The Unreasonable Effectiveness of Philosophy
It seems like human philosophy is more effective than it has any right to be. Why?
First I'll try to establish that there is a mystery to be solved. It might be surprising so see the words "effective" and "philosophy" together in the same sentence, but I claim that human beings have indeed made a non-negligible amount of philosophical progress. To cite one field that I'm especially familiar with, consider probability and decision theory, where we went from having no concept of probability, to studies involving gambles and expected value, to subjective probability, Bayesian updating, expected utility maximization, and the Turing-machine-based universal prior, to the recent realizations that EU maximization with Bayesian updating and the universal prior are both likely to be wrong or incomplete.
We might have expected that given we are products of evolution, the amount of our philosophical progress would be closer to zero. The reason for low expectations is that evolution is lazy and shortsighted. It couldn't possibly have "known" that we'd eventually need philosophical abilities to solve FAI. What kind of survival or reproductive advantage could these abilities have offered our foraging or farming ancestors?
From the example of utility maximizers, we also know that there are minds in the design space of minds that could be considered highly intelligent, but are incapable of doing philosophy. For example, a Bayesian expected utility maximizer programmed with a TM-based universal prior would not be able to realize that the prior is wrong. Nor would it be able to see that Bayesian updating is the wrong thing to do in some situations.
Why aren't we more like utility maximizers in our ability to do philosophy? I have some ideas for possible answers, but I'm not sure how to tell which is the right one:
- Philosophical ability is "almost" universal in mind space. Utility maximizers are a pathological example of an atypical mind.
- Evolution created philosophical ability as a side effect while selecting for something else.
- Philosophical ability is rare and not likely to be produced by evolution. There's no explanation for why we have it, other than dumb luck.
As you can see, progress is pretty limited so far, but I think this is at least a useful line of inquiry, a small crack in the problem that's worth trying to exploit. People used to wonder at the unreasonable effectiveness of mathematics in the natural sciences, especially in physics, and I think such wondering eventually contributed to the idea of the mathematical universe: if the world is made of mathematics, then it wouldn't be surprising that mathematics is, to quote Einstein, "appropriate to the objects of reality". I'm hoping that my question might eventually lead to a similar insight.
Objective Philosophical Truths?
Consider again the example of the wrongness of the universal prior and Bayesian updating. Assuming that they are indeed wrong, it seems that the wrongness must be objective truths, or in other words, it's not relative to how the human mind works, or has anything to do with any peculiarities of the human mind. Intuitively it seems obvious that if any other mind, such as a Bayesian expected utility maximizer, is incapable of perceiving the wrongness, that is not evidence of the subjectivity of these philosophical truths, but just evidence of the other mind being defective. But is this intuition correct? How do we tell?
In certain other areas of philosophy, for example ethics, objective truth either does not exist or is much harder to find. To state this in Eliezer's terms, in ethics we find it hard to do better than to identify "morality" with a huge blob of computation which is particular to human minds, but it appears that in decision theory "rationality" isn't similarly dependent on complex details unique to humanity. How to explain this? (Notice that "rationality" and "morality" otherwise share certain commonalities. They are both "ought" questions, and a utility maximizer wouldn't try to answer either of them or be persuaded by any answers we might come up with.)
These questions perhaps offer further entry points to try to attack the larger problem of understanding and mechanizing the process of philosophy. And finally, it seems worth noting that the number of people who have thought seriously about meta-philosophy is probably tiny, so it may be that there is a bunch of low-hanging fruit hiding just around the corner.
Arguably, OCR is about taking a small patch of an image and matching that to a finite set of candidate possible ground truths. OCR programs can do this sometimes better than most humans, if the only thing you look at is one distorted character.
OCR has traditionally been a difficult problem and there are some novel applications of statistics and heuristics used to solve it. But OCR is not what we actually care about: the problem is recognizing a document, or symbolically representing a sentence, and OCR is just one small problem we've carved out to help us deal with the larger problem.
Characters are important when they are part of words, and the structure of a document. They are important when they contribute to what the document means, beyond just the raw data of the image scan. Situating a character in the context of the word it's in, the sentence that word is in, and the context of the document (English novel, handwritten letter from the 18th century, hastily scribbled medical report from a German hospital in 1970's) is what allows a human to extrapolate what the character must be, even if the image of the original character is distorted beyond any algorithm's ability to recognize, or even obliterated entirely.
It's this effect of context which is hard to capture and encode into an OCR algorithm. This broader sense, of being able to recognize a character anywhere a human would, which is the end goal of the problem, is what my friends refer to as an AI-complete problem. (Apologies if this community also uses that phrase, I haven't yet seen it here on LW.)
To give a specific example, many doctors use the symbol "circle above a cross" to indicate female, which most people reading would understand. Why? We've seen that symbol before, perhaps many times, and understand what it means. If you've trained your OCR algorithm on the standard set of English alphanumeric characters, then it will attempt to match that symbol and come up with the wrong answer. If you've done unsupervised training of an OCR algorithm on a typical novel, magazine, and newspaper corpus, there is a good chance that the symbol for female does not appear as a cluster in its vector space.
In order to recognize that symbol as a distinct symbol that needs to be somehow represented in the output, an OCR algorithm would have to do unsupervised online learning as it's scanning documents in a new domain. Even then, I'm not sure how useful it would be, since the problem is not recognizing a given character. The problem is recognizing what that character should be given the context of the document you're scanning. The problem of OCR explodes into specializations of "OCR for novels, OCR for 18th century English letters, OCR for American hospitals", and even more.
If we want an OCR algorithm to output something more useful than [funky new character I found], and instead insert "female" into the text database, at some point we have to tell the algorithm about the character. There's not yet that I know of an OCR system that avoids this hard truth.
I like "AI-complete", though it wouldn't surprise me if general symbol recognition and interpretation is easier than natural language, whereas all NP-complete problems are equivalent.