Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Alex_Altair comments on Computation Hazards - Less Wrong Discussion

13 Post author: Alex_Altair 13 June 2012 09:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (57)

You are viewing a single comment's thread. Show more comments above.

Comment author: JGWeissman 13 June 2012 10:32:20PM 0 points [-]

There are other hazards that may arise in the course of running large-scale computations.

Every computational hazard I can come up with involves simulating people. Do you have examples of any that don't?

Comment author: Alex_Altair 13 June 2012 10:56:42PM 1 point [-]

Yeah, the "self-improving agent", "simulate powerful agents", "self-fulfilling prophecies" and "oracles or predictors may become agents" were all meant to be examples of computation hazards, and those doesn't necessarily involved simulating people.

Comment author: JGWeissman 13 June 2012 11:18:39PM 2 points [-]

Ah, I was thinking of "computational hazard" as meaning the computation itself is bad, not its consequences on the computing substrate or outside environment. I thought a "self-improving agent" was an example of something that might compute a hazard as a result of computing lots of stuff, some of which turns out to be hazardous. But short of instantiating that particular computational hazard, I don't think it does bad merely by computation, rather the computation helps it direct its actions to achieve bad consequences.

Comment author: Alex_Altair 14 June 2012 12:19:07AM 0 points [-]

I think I agree.