Comment on Fusing AI with Superstition

Wed, 21 Apr 2010 21:17:05 GMT

In particular, it might be impossible to construct a simple action description which will fit all of human future. However, it is certainly not harder than to construct a real moral system.

"Certainly" is a lullaby word (hat-tip to Morendil for the term), and a dangerous one at that. In this case, your "certainly" denies that anyone can make the precise objection that everyone has been making.

FAI theory talks a lot about this kind of thinking - for example, I believe The Hidden Complexity of Wishes was specifically written to describe the problem with the kind of thinking that comes up with this idea.

Discuss