Would someone familiar with the topic be able to do a top level treatment similar to the recent one on self-help? A survey of the literature, etc.
I am a software engineer, but I don't know much about general artificial intelligence. The AI research I am familiar with is very different from what you are talking about here.
Who is currently leading the field in attempts at providing mathematical models for philosophical concepts? Are there simple models that demonstrate what is meant by computational meta-ethics? Is that a correct search term -- as in a term that I could key into google and get meaningful results? What are the correct search terms? I see lots of weird and varied results when I search on "computational meta-ethics" and "algorithmic epistemology" and other combinations. There is no popular set of references for this (like there would be for psychological terminology) so I don't even have a pop-culture-like values to use to evaluate the meaning of search results.
Does the language above represent the current approach to the problem? In other words, is the process currently employed to try and reduce via linguistic methods human ethical concepts into algorithmic models? That seems incredibly dangerous. Are there other approaches? Can someone detail the known methods that are being used, their most successful implementors, and references to examples?
I have read that intelligence can be viewed as a type of optimization process. There is a set of detailed research around optimization processes (genetic algorithms, etc). Is there AI research in this area? I don't have the language to describe what I think I mean.
I do know that there is very high quality, useful research being done on directed optimization processes. I could sit down with this research, read it, and formulate new research plans to progress it in a very rigorous and detailed way. Much of this stuff is accessible to moderately capable software engineers (Haupt, 2004). I don't see that with general AI because I don't know where to look. A post that helps me know where to look would be great.
What are the works that I need to internalize in order to begin to attack this problem in a meaningful way?
It sounds like you're asking for something broader than this, but I did just post a bibliography on Friendly AI, which would make for a good start.
Unfortunately, meta-ethics is one of the worst subjects to try to "dive into," because it depends heavily on so many other fields. I was chatting with Stephen Finlay, a meta-ethicist at USC, and he said something like: "It's hard to have credibility as a professor teaching meta-ethics, because meta-ethics depends on so many fields, and in most of the graduate courses I teach on meta-ethics, I know that every one of my students knows more about one of those fields than I do."
Barring a major collapse of human civilization (due to nuclear war, asteroid impact, etc.), many experts expect the intelligence explosion Singularity to occur within 50-200 years.
That fact means that many philosophical problems, about which philosophers have argued for millennia, are suddenly very urgent.
Those concerned with the fate of the galaxy must say to the philosophers: "Too slow! Stop screwing around with transcendental ethics and qualitative epistemologies! Start thinking with the precision of an AI researcher and solve these problems!"
If a near-future AI will determine the fate of the galaxy, we need to figure out what values we ought to give it. Should it ensure animal welfare? Is growing the human population a good thing?
But those are questions of applied ethics. More fundamental are the questions about which normative ethics to give the AI: How would the AI decide if animal welfare or large human populations were good? What rulebook should it use to answer novel moral questions that arise in the future?
But even more fundamental are the questions of meta-ethics. What do moral terms mean? Do moral facts exist? What justifies one normative rulebook over the other?
The answers to these meta-ethical questions will determine the answers to the questions of normative ethics, which, if we are successful in planning the intelligence explosion, will determine the fate of the galaxy.
Eliezer Yudkowsky has put forward one meta-ethical theory, which informs his plan for Friendly AI: Coherent Extrapolated Volition. But what if that meta-ethical theory is wrong? The galaxy is at stake.
Princeton philosopher Richard Chappell worries about how Eliezer's meta-ethical theory depends on rigid designation, which in this context may amount to something like a semantic "trick." Previously and independently, an Oxford philosopher expressed the same worry to me in private.
Eliezer's theory also employs something like the method of reflective equilibrium, about which there are many grave concerns from Eliezer's fellow naturalists, including Richard Brandt, Richard Hare, Robert Cummins, Stephen Stich, and others.
My point is not to beat up on Eliezer's meta-ethical views. I don't even know if they're wrong. Eliezer is wickedly smart. He is highly trained in the skills of overcoming biases and properly proportioning beliefs to the evidence. He thinks with the precision of an AI researcher. In my opinion, that gives him large advantages over most philosophers. When Eliezer states and defends a particular view, I take that as significant Bayesian evidence for reforming my beliefs.
Rather, my point is that we need lots of smart people working on these meta-ethical questions. We need to solve these problems, and quickly. The universe will not wait for the pace of traditional philosophy to catch up.