Does agency enable the AI to do so? If not, then why wouldn't a human being not be able to do the same by using the AI in tool mode?
Yeah, I realized this while writing the comment: "(Or another human with superintelligent tools could do this as well.)" So this isn't a risk with self-willed AI per se. But note this actually makes my original point stronger, since I was arguing against the idea that progress on AI is safe as long as it doesn't have a "will" to act in the real world.
Just make it list equally convincing counter-arguments.
So every time you look at a (future equivalent of) website or email, you ask your tool to list equally convincing counter-arguments to whatever you're looking at? What does "equally convincing" mean? An argument that exactly counteracts the one that you're reading, leaving your mind unchanged?
So every time you look at a (future equivalent of) website or email, you ask your tool to list equally convincing counter-arguments to whatever you're looking at?
Sure, why not? I think IBM is actually planning to do this with IBM Watson. Once mobile phones become fast enough you can receive constant feedback about ideas and arguments you encounter.
For example, some commercial tells you that you can lose 10 pounds in 1 day by taking a pill. You then either ask your "IBM Oracle" or have it set up to give you automatic feedback. It will then tel...
Why does SI/LW focus so much on AI-FOOM disaster, with apparently much less concern for things like
Why, for example, is lukeprog's strategy sequence titled "AI Risk and Opportunity", instead of "The Singularity, Risks and Opportunities"? Doesn't it seem strange to assume that both the risks and opportunities must be AI related, before the analysis even begins? Given our current state of knowledge, I don't see how we can make such conclusions with any confidence even after a thorough analysis.
SI/LW sometimes gives the impression of being a doomsday cult, and it would help if we didn't concentrate so much on a particular doomsday scenario. (Are there any doomsday cults that say "doom is probably coming, we're not sure how but here are some likely possibilities"?)