= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on Freaky Fairness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (36)
This is the kind of issue that makes me skeptical about FAI. We can teach a decision algorithm to an AI, but can we teach the AI how to think about decision algorithms? How to decide whether one concept of fairness is better than another? Whether one interpretation of probability theory is better than another? Whether one Bayesian prior is better than another? To see the obviousness that coding conventions shouldn't matter?
If we can't, if we have to hard code all these answers into the AI, then that's hardly safe, since it seems unlikely that we (i.e. ordinary humans) can achieve a high confidence in all of the answers. But as far as I know, no one has attempted or even proposed to formalize the process of philosophy as an algorithm.
These are old questions, and certainly they'll need to be answered. There is no running away from having a good theory that explains why the take-off process can be trusted to do the right thing. Ideally, this theory will be as clear-cut as classical mechanics.
The data is in the process of humanity, but so is the right way of interpreting it. How much do you need to specify by hand to launch the process of extracting preference of humanity formally, that knows what to look for, and is able to find enough to continue looking for more? It'd take at least a technical theory of preference generalized to non-formal agents (of which the humanity is an instance) to answer that question.