fwang

Honesty is the best policy. <3 Mom

Regarding the assumptions for a strong AI that will cease cooperating and pursue its own values, the scenario presented assumed that 1) L will have the ability to subvert S's control, and then 4a) S must be unaware that L has that ability. That is (if I understand this correctly), the ability for L to subvert S is undetected. But if it's the case that we assume 1), then perhaps S should instead operate under the assumption that L already has the ability (and perhaps even knowledge) to subvert, but S doesn't know what exactly this ability is, and if S knew, not the knowledge of... (read more)

This may be a little basic, but if you're already going with the "efficient at transmitting forces" idea for physical posture, then I think a good analogy in terms of mental posture would simply be "efficient at processing information" (of which being rational is a pretty useful method, just like keeping your spine, er, non-kyphotic is useful).

This is much more concise than:

how efficiently one's patterns of directing attention let one mentally navigate one's environment

while at the same time relatively neutral with respect to the actual goals that you may have (e.g. keeping one's balance, either physically when someone bumps into you, or mentally when someone offends you).

And if you were to compare... (read more)