James_Miller comments on Crazy Ideas Thread - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (344)
I trust it to understand my intent based on everything else it can learn about me. Plus, there is probably nothing I could write that would protect me against a not-quite-benevolent genie who wanted to harm me but was constrained by the literal meaning of what I wrote.
True, but the interesting question is whether you want to summon a genie of uncertain benevolence.
I wouldn't summon a genie of uncertain benevolence. But if one already exists it might be worth asking it for a favor. Most unfriendly genies would ignore my wishes. The question is, do you want to draw the attention of a genie of uncertain benevolence that is influenced by your explicit requests to it?
You have heard it said, "Do not call up that which you cannot put down."
But I tell you, "That's how we got here."