I meant that not only his post but most of his comments were downvoted, and from my personal experience if I get a lot of downvoting, I feel difficult to continue rational discussion of the topic.
Egan's law is very vague in its short formulation. It is not clear, what is "all", what kind of law is it - epistemic, natural, legal; what is normality - physics, experience, our expectation, our social agreements. So it mostly used as universal objection to any strange things.
But there are lot of strange things. Nukes were not normal before they were created, and if one apply Egan's law before their creation, he may claim that they are not possible. Strong self improving AI also is something new on Earth, but we don't use Egan's law to disprove its possibility.
Your interpretation of Egan's law is that everything useful should already be used by evolution. In case of QI it has some similarities to anthropic principle, by the way, so there is nothing new here from evolutionary point of view.
You also suggest to use Egan's law as normative: don't do strange risky things.
I could suggest more correct formulation of Egan's law: it all adds up to normality in local surroundings (and in normal circumstances.)
And from this follows that than surrounding become large enough everything is not normal (think about black holes, sun became red giant, or strange quantum effects in small scale)
In local surrounding Newtonian, relativistic and quantum mechanics produce the same observations and the same visible world. Also in normal circumstances I will not put a bomb into my house.
But, as OP suggested, I know that soon 1 of 16 outcomes will happen, where 15 will kill the Earth and me, so my best strategy should not be normal. In this case going into a submarine with a diverse group of people capable to restore civilization may be best strategy. And here I get benefits even if QI doesn't work, so it positive sum game.
I put only 10 per cent probability in QI to work as intended, so I will try any other strategy which have higher payoff (if I have any). That is why I will not put a bomb under my house in normal situations.
But there are situations there I don't risk anything if I use QI, but benefit if it works. One of them is cryonics, to which I signed up.
So it mostly used as universal objection to any strange things.
Well, for the avoidance of doubt, I do not endorse any such use and I hope I haven't fallen into such sloppiness myself.
Your interpretation of Egan's law is that everything useful should already be used by evolution.
No, I didn't intend to say or imply that at all. I do, however, say that if evolution has found some particular mode of thinking or feeling or acting useful (for evolution's goals, which of course need not be ours) then that isn't generally invalidated by new discoveries abou...
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.