Dorikka comments on Open Thread: September 2011 - LessWrong

5 Post author: Pavitra 03 September 2011 07:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (441)

You are viewing a single comment's thread. Show more comments above.

Comment author: Dorikka 04 September 2011 07:06:53PM *  0 points [-]

I think that humans can mentally self-modify to some extant, especially if it really really matters. If you really needed to be optimistic, you might be able to modify yourself to be such by significantly participating in certain types of organized religion. (This is a rather extreme example -- a couple minutes of brainstorming would probably yield ideas with (much?) lower cost and similar results, but it illustrates the possibility.)

Expected utility maximizers are not necessarily served by updating their map to accurately reflect the territory -- there are cases such as the above when one might make an effort to willingly make one's map reflect the territory less accurately. The reason why expected utility maximizers often do try to update their map to accurately reflect the territory is that it usually yields greater utility in comparison to alternative strategies -- having an accurate map is (I would guess) not much of a source of terminal utility for most.

ETA: Missing words. >.<

Comment author: Armok_GoB 04 September 2011 09:01:05PM 0 points [-]

I might theoretically be able to do this, but it would involve rejecting the entirely of rationality and becoming a sophilist or somehting, so after recovery the thing my body would have become would not undo the modification and instead go intentionally create UFAI as an artistic statement or somehting.

Ok, a slight exaggeration, but far less slight than I'm comfortable with.

Comment author: Dorikka 04 September 2011 11:18:52PM 1 point [-]

Since you're likely the one who would benefit from it, hopefully you brainstormed for a few minutes before you decided that my "religion" approach was really the most effective one -- I just typed the first idea that popped in my head and seemed to work.

Comment author: Armok_GoB 05 September 2011 11:05:56AM 0 points [-]

Huh? Not only was it just an example, but Sophilism is incompatible with every religion I know of.

Anyway, I didn't brainstorm it for roughly the same reason I don't brainstorm specific ways to build a pepertum mobile. The way my brain is set up, I can't reject rationality in any single situation like that without rejecting the entire concept of rationality, and without that my entire belief structure disintegrates onto postmodern relativist sophilism. Similar but more temporary things have happened before and the consequences are truly catastrophic.

And yea, this obviously isn't how it's supposed to work but I've not been able to fix it, or even figure out what would be needed to do so.