Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Psy-Kosh comments on Devil's Offers - Less Wrong

21 Post author: Eliezer_Yudkowsky 25 December 2008 05:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (46)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Psy-Kosh 25 December 2008 05:52:08PM 2 points [-]

Hrm... I think, at least initially I'd want some limiters for myself along the lines of, well, the system telling me "this isn't going to do what you actually really want it to do, so no."

But "no mental alteration at all without being a neuroanatomical master yourself in the first place", at least initially, seems a bit too harsh. That is, to the extent that one needs a bit of an intel/etc boost to fully master it in the first place, we'd have a bit of a problem here. :)

I'd be perfectly happy with something that, if, say, I said "I'd like to lower my agression" and it came back with "uh, no. The structure of your mind is such that that is tied to ambition/get-it-doneness and so on, and since even by your own measure you're overly passive as is. This is not what you want, even if you think it is."

(Note, I'm not saying that I have the knowledge to, well, say that agression and ambition are tied to each other like that. This is just a hypothetical, though at least from personal introspection they do seem potentially related like that)

But I'd like it if it also add on something like "However here is a subtler change that can be made that would have the effects you actually wanted out of what you just asked for."

or even "And on that note, here's something to do about that passivity/laziness that seems to be something that is a much larger source of frustration on your part."

However, I don't really have any objection to it sometimes returning with "No. This is the sort of thing you really want (even if you don't know it) to do/work out for yourself in terms of what's already available to you."

And on the other other other hand, there's the issue of "do we really want it to be the sort of thing that we'd perceive as a person, rather than an abstract process?"