What is the point of this post? I seem to have missed it entirely. Can anyone help me out?

The mirror challenge for decision theory is seeing which option a choice criterion really endorses. If your stated moral principles call for you to provide laptops to everyone, does that really endorse buying a $1 million gem-studded laptop for yourself, or spending the same money on shipping 5000 OLPCs?

Is the point that predicting the end result of particular criterion is difficult because bias gets in the way? And, because it is difficult, start small with stuff like gene fitness and work up to bigger problems like social ethics?

Where is the pure moral reasoner [...] whose trustworthy output we can contrast to human rationalizations of the same utility function? [...] Why, it's our old friend the alien god, of course! Natural selection is guaranteed free of all mercy, all love, all compassion, all aesthetic sensibilities, all political factionalism, all ideological allegiances, all academic ambitions, all libertarianism, all socialism, all Blue and all Green.

Or... is the point that natural selection is a great way to expose the biases at work in our ethics choice criterion?

I am not tracking on something here. This is a summary of the points in the post as I see them:

• We are unable to accurately study how closely the results of our actions match our own predictions of those results.

• The equivalent problem in decision theory is that we are unable to take a set of known choice criteria and predict which choice will be made given a particular environment. In other words, we think we know what we would/should do in event X but we are wrong.

• We possess the ability to predict any particular action from all possible choice criteria.

• Is it possible to prove that a particular action does or does not follow from certain choice criteria, thereby avoiding our tendency to predict anything from everything?

• We need a bias free system to study that allows us to measure our predictions without interfering with the result of the system.

• Natural selection presents a system whose only "goal" is inclusive genetic fitness. There is no bias.

• Examples show that our predictions of natural selection reveal biases in ourselves. Therefore, our predictions were biased.

• To remove our bias with regards to human ethics, we should use natural selection as a calibration tool.

I feel like the last point skipped over a few points. As best as I can tell, these belong just before the last point:

• When our predictions of the bias-proof system are accurate, they will be predictions without bias.

• Using the non-biased predictors we found to study the bias-proof system, we can study other systems with less bias.

Using this outline, it seems like the takeaway is, "Don't study ethics until after you studied natural selection because there is too much bias involved in studying ethics."

Can someone tell me if I am correct? A simple yes or no is cool if you don't feel like typing up a whole lot. Even, "No, not even close," will give me more information than I have right now.

As a counterpoint, my highest rated comments are huge walls of text. This could be because (a) I don't make a lot of jokes or (b) I make crappy jokes or (c) people like my walls of text more than the typical wall of text or (d) something else.

I keep an eye on my karma and have noticed these things that I believe are related to your post:

• Talking about the karma system has fallen out of favor. I think people are getting tired of it.

• Asking why something was downvoted usually brings more upvotes unless you really, really deserved the downvotes. This latter case will probably be swarmed with downvotes.

• Some (many?) people vote with an end score in mind. These meta-voters have more affect on threshold comments that fluctuate between -2 and 2.

• Long conversations will generally pull between -1 and +2 per post. Most of my karma comes from lengthy discussions in the comments. Even if the top level post was only rated +2 I will net almost 100 karma points from the post and comments. (Note: I haven't actually added this up. It may be closer to 75 karma.)

• Quick responses pointing out third alternatives or simple problems get upvoted and usually roam between +2 and +14. If you want to get your karma higher, this is the easiest way. Comment immediately after a post is submitted and point out the most obvious flaw respectfully and concisely. Don't try to make a point, just note an error. If you get in before the rest of us, you will probably get +4 or higher.

• Long responses pointing out serious problems get upvoted but have a lower chance of pulling large amounts of karma than quick responses. However, after the first wave of quick responses, only the longer comments are true candidates for higher karma. I suspect that a long response to a quick comment has a good chance to do well, but haven't really watched those comments yet.

• Jokes are upvoted when they are either extremely funny or solidly funny and on topic. Randomness is upvoted if it is an inside joke, otherwise stays around +0. Sarcasm is appreciated but has the problem of being mistaken for nonsarcasm.

• Extending the point or conversation of a top-level post gets upvoted. Most of mine get between +2 and +4. Examples would be almost every comment I have made while reading the sequences.

• Aggressiveness is generally poorly received on technical topics. Aggressiveness is easier to get away with when dealing with fuzzy topics. I attribute this mostly to margin of error. Technical topics are harder to be bulletproof. I am still having a hard time predicting which non-technical top level posts are voted up. I suspect this is because I don't know what is already been discussed or that the issue is a technical topic that I have misidentified.

• Self-depreciation is a huge karma pull. Both my highest rated post and comment were essentially me slamming myself over and over. Each were voted higher than +20.

• Bullet point lists of extensions, ideas, questions, and so on seem to do about as well or better than long paragraphs of text. The walls of text may be harder to skim for goodies or the bullet point list better organized?

• Non-aggressive requests for clarification or information are not generally downvoted. Mine seem to roam between 0 and +2 karma. If the question and response delve into a lengthy but friendly conversation, I seem to get between 0 and +2 karma for each of my comments. If a solid agreement or conclusion is reached, the capping comment gets about double whatever the individual comments were getting.

• Posting nearby "famous" people amps up the karma action. Replying to EY, Alicorn, Vladimir_Nesov, pjeby, et al will increase the amount of people that read your comments. The reasons for this are varied. The four I used here are just names that popped into my head. Also, some people seem to vote up conversations they are in while others do not. A few downvote anyone disagreeing with them. The people that matter generally fit these criteria: (a) top contributer (b) easily recognizable name (c) frequent poster/commenter (d) abnormal amount of recent activity (e) holds atypical beliefs for the community (f) a troll.

• Better grammar, spelling, and language increases the likelihood that your comment will move upward faster.

• Comments quickly upvoted higher than typical seem to either (a) go through the roof or (b) get meta-voted back to between +1 and +3.

• Telling people to vote in a particular way tends to produce easy to predict results but not in a manner that is easy to describe.

And oh wow did that get long. Do note that this is all being typed from the top of my head using myself and the comments I read as an example. Naturally, the above does not dictate how people vote.

EDIT: I guess for fun, I predict that this comment gets ROT13orgjrra cbfvgvir gjb naq cyhf fvk xneznROT13.

EDIT 2: I would normally downvote a post such as this but elsewhere in the comments you seemed to have received the message and was wondering about deleting it. So I just left it as it is. Also relevant: I generally do not upvote jokes unless they are truly amazing.

In his rant against intelligent design theory , Yudkowsky seems to have overlooked a simple fact. Darwinian Evolution is irrelevant to the whole discussion. Darwinian Evolution is only operative and relevant from the moment you have a DNA based organism capable of self replication. (I know that there are highly speculative theories of earlier "simpler" self replicating molecules. There is no evidence at all that they ever actually existed, and no one has ever seen one outside of a laboratory where even the highly limited ability to self replicate is a product of the intelligent design of the chemists and microbiologists involved)

Since absolutely no one has ever come up with anything even approaching a plausible naturalistic explanation of the origin of life from non-life, the obvious truth is that the first DNA based bacterium (the simplest life form we know of) with it's staggeringly functionally complex digital code was created by a supernatural intelligence.

Again, all forms of life are possible (I.e. Darwinian Evolution and Natural Selection are possible) if, and only if, the proper molecular machinery is in place. The irony is that not only is Darwinian Evolution not an explanation nor the cause of the fantastic and astounding functional complexity of life on this planet, Darwinian Evolution is a process that is the result of the astounding functional complexity of life on this planet.

An argument isomorphic to yours can be used to demonstrate that spousal cheating is okay as long as there are no consequences and the spouse doesn't know. Maybe your concept of "valid objection" is overly narrow?

I find it hard to believe that posts about the collapse postulate are on-topic but this one is not. Is there a substantial difference between the two that I'm missing?

As a suggestion, maybe typos that have no substantial impact on readability should be communicated to the author through a direct message rather than a public comment.

EDIT: For historical purposes, this comment reported two typos that have since been fixed. I was intending to delete this comment when they were fixed but a valuable discussion occurred below.

So, when I pester them for a rational reason, why do they keep giving an answer that is irrational for this situation?

Because human beings often first have a reaction based on an evolved, unconscious heuristic, and only later form a conscious rationalization about it, which can end up looking irrational if you ask the right questions (e.g. the standard reactions to the incest thought experiment there). So, yes, they were probably unaware of the heuristic they were actually using.

I'd suppose that the heuristic is along the lines of the following: Say there's an agreed-upon fair procedure for deciding who gets something, and then someone changes that procedure, and someone other than you ends up benefiting. Then it's unfair, and what's yours has probably been taken.

Given that rigorous probability theory didn't emerge until the later stages of human civilization, there's not much room for an additional heuristic saying "unless it doesn't change the odds" to have evolved; indeed, all of the agreed-upon random ways of selecting things (that I've ever heard of) work by obvious symmetry of chances rather than by abstract equality of odds†, and most of the times someone intentionally changed the process, they were probably in fact hoping to cheat the odds.

† Thought experiment: we have to decide a binary disagreement by chance, and instead of flipping a coin or playing Rock-Paper-Scissors, I suggest we do the following: First, you roll a 6-sided die, and if it's a 1 or 2 you win. Otherwise, I roll a 12-sided die, and if it's 1 through 9 I win, and if it's 10 through 12 you win.

Now compute the odds (50-50, unless I made a dumb mistake), and then actually try it (in real life) with non-negligible stakes. I predict that you'll feel slightly more uneasy about the experience than you would be flipping a coin.

When I was young, I happened upon a book called "The New Way Things Work," by David Macaulay. It described hundreds of household objects, along with descriptions and illustrations of how they work. (Well, a nuclear power plant, and the atoms within it, aren't household objects. But I digress.) It was really interesting!

I remember seeing someone here mention that they had read a similar book as a kid, and it helped them immensely in seeing the world from a reductionist viewpoint. I was wondering if anyone else had anything to say on the matter.

If they raped you, starved you/fed you paint chips, beat you to the point of brain injury, tortured you? How about being born in a place where the pollution is so bad that you're likely to get sick/die from with a very high probability? Places that are completely ravaged with drought or famine? Places where genocide is fairly regular? Where your parents are so destitute that they are forced to feed you the absolute worst food (or even non-"food") so that your brain/body never develops properly?

Of course, for people/places where rape/forced childbirth is prevalent or the knowledge of how pregnancy occurs is still non-existent, it's understandable. For places where the former isn't and the latter is, there really should be no statute of limitations on blame.

The quote is good, but should be understood to apply only in certain contexts (i.e., to people who weren't born into horrific conditions and who live(d) in a place with something resemble equality of opportunity.) Not understanding this perpetuates the idea that "everything that happens to you is your own fault" that appears in some popular strains of political thought today, when it clearly cannot be universally applied.