= 27d567bc2d48d9f40e9ccc20ea370b15)
Will_Newsome comments on Branches of rationality - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (64)
One trick that we can always apply: disassemble the human and use his atoms to build a paperclip maximizer. The point is, we don't just want to turn humans into coherent agents, we want to turn them into coherent agents who can be said to have the same preferences as the original humans. But given that we don't have a theory of preferences for incoherent agents, how do we know that any given trick intended to improve coherence is preference-preserving? Right now we have little to guide us except intuition.
To borrow an example from Robin Hanson, we have both preferences that are consciously held, and preferences that are unconsciously held, and many "rationality techniques" seem to emphasize the consciously held preferences at the expense of unconsciously held preferences. It's not clear this is kosher.
I think there are many important unsolved problems in the theoretical/philosophical parts of rationality, and this post seems to under-emphasize them.
Agreed to an extent, but most folk aren't out to become Friendliness philosophers. One branch that went unmentioned in the post and would be useful for both philosophers and pragmatists includes the ability to construct (largely cross-domain / overarching) ontologies out of experience and abstract knowledge, the ability to maintain such ontologies (propagating beliefs across domains, noticing implications of belief structures and patterns, noticing incoherence), and the disposition of staying non-attached to familiar ontologies (e.g. naturalism/reductionism) and non-averse to unfamiliar/enemy ontologies (e.g. spiritualism/phenomenology). This is largely what distinguishes exemplary rationalists from merely good rationalists, and it's barely talked about at all on Less Wrong.