Desrtopa comments on No Universally Compelling Arguments in Math or Science - Less Wrong

30 Post author: ChrisHallquist 05 November 2013 03:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (227)

You are viewing a single comment's thread. Show more comments above.

Comment author: Desrtopa 05 November 2013 09:25:43PM *  0 points [-]

One of the key disanalogies between your "ecosystem" formulation and human development of technology is that natural selection isn't an actor subject to feedback within the system.

If an organism develops a mutation which is sufficiently unfavorable to the Blind Idiot God, the worst case scenario is that it's stillborn, or under exceptional circumstances, triggers an evolution to extinction. There is no possible failure state where an organism develops such an unfavorable mutation that evolution itself keels over dead.

However, in an ecosystem where multiple species interrelate and impose selection effects on each other, then a sudden change in circumstances for one species can result in rapid extinction for others.

We impose selection effects on technology, but a sudden change in technology which kills us all would not be a novel occurrence by the standards of ecosystem operation.

ETA: It seems that your argument all along has boiled down to "We'll just deliberately not do that" when it comes to cases of catastrophic failure. But the argument of Eliezer and MIRI all along has been that such catastrophic failure is much, much harder to avoid than it intuitively appears.