(Of course, if there is a way to quantify the marginal benefit of an LW post, then there is also a way to quantify the marginal cost from a bad one--just reverse the sign, and you'll be right on average.)

Huh? Why should these be equal? Why should they even be on the same order of magnitude? For example, an advertising spam post that gets deleted does orders of magnitude much less harm than an average good post does good. And a post that contained designs for a UFAI would do orders of magnitude more harm.

(Of course, if there is a way to quantify the marginal benefit of an LW post, then there is also a way to quantify the marginal cost from a bad one--just reverse the sign, and you'll be right on average.)

That makes sense for evaluating the cost/benefit to me of reading a post. But if I want to evaluate the overall cost/benefit of the post itself, I should also take into account the number of people who read one vs. the other. Given the ostensible purpose of karma and promotion, these ought to be significantly different.

I have not seen the original post, but can't someone simply post it somewhere else? Is deleting it from here really a solution (assuming there's real danger)? BTW, I can't really see how a post on a board can be dangerous in a way implied here.

The problem is, are you people sure you want to take this route? If you are serious about all this, what would stop you from killing a million people if your probability estimates showed that there was a serious risk posed by those people?

If you read this comment thread you'll see what I mean and what danger there might be posed by this movement, 'follow Eliezer', 'donating as much as possible to SIAI', 'kill a whole planet', 'afford to leave one planet's worth', 'maybe we could even afford to leave their brains unmodified'...lesswrong.com sometimes makes me feel more than a bit uncomfortable, especially if you read between the lines.

Yes, you might be right about all the risks in question. But you might be wrong about the means of stopping the same.

This really isn't worth arguing and there isn't any reason to be angry...

You are wrong on both. There is strong signalling going on that gives good evidence regarding both Eliezer's intent and his competence.

What Roko said matters little, what Eliezer said (and did) matters far more. He is the one trying to take over the world.

I don't consider frogs to be objects of moral worth. -- Eliezer Yudkowsky

Yeah ok, frogs...but wait! This is the person who's going to design the moral seed of our coming god-emperor. I'm not sure if everyone here is aware of the range of consequences while using the same as corroboration of the correctness pursuing this route. That is, are we going to replace unfriendly AI with unknown EY? Are we yet at the point that we can tell EY is THE master who'll decide upon what's reasonable to say in public and what should be deleted?

Ask yourself if you really, seriously believe into the ideas posed on LW, enough to follow into the realms of radical oppression in the name of good and evil.