The two links to an article on Solving The Wrong Problem found in the original are dead. I'm doubtful of that article having much of value to add to what's right on the tin, but in case it did (or simply for the sake of completeness): does anyone know where it could be found? Googling the title returns thousands of hits, some of them blog posts by the same name by various authors.

I wish there was a "subscribe to all" button.

Why is the top-level comment retracted?

Even weirder are the ones who know the math, agree with you that something is a good bet for them to take, and then refuse to bet anyway!

I know a number of mathematically literate people who buy lottery tickets. Their usual justification is that they pay for happiness provided by the hope of winning.

Um. Many of those high-status behaviours sound pretty rude. Others lead to low epistemic hygeine. If we start behaving in those ways to each other, it won't work out. It's hard to be confident about these things, but people say that I manage to come across as confident about myself without doing other people down, which is certainly both what I aim for and how I feel. I'd like to imagine that I'd be happy if other people behaved the way I do. It's hard to hit that target - indeed, hard to know whether you've hit it or not - but it seems the right one to aim for.

EDIT: at least one person seems to have carried away the impression I'm saying that all of those behaviours are always rude and should always be avoided. That definitely isn't the case; I think it's nearly always rude to interrupt before you know what you're going to say, but I'm not, for example, against speaking in complete sentences.

Posted this above as well.

The reason they all feel like babies to me, from the perspective of "are they people?", is that they're in a condition where we can see a reasonable path for turning them into something that is unquestionably a person.

Here's another case to consider:

I assume you've granted that sufficiently advanced AIs ought to be counted as people. Say that I have running on my computer a script which is compiling an AI's source, and which will launch the resultant executable as soon as compilation finishes with no intervention on my part.

Am I killing a person if I terminate this script before compilation completes? That is, does "software which will compile and run an AI" belong to the "people" or the "not people" group?

I think babies are much closer to this than to any of the examples you've listed above.

In the interests of settling confusion, here's another example:

Suppose we let the above script finish and the AI go about its merry way for a few centuries. We shut down the computer it's running on - writing its current state to non-volatile memory - to transport it somewhere else. To me it seems that destroying that memory would constitute killing a person.

From these examples, I think "will become a person" is only significant for objects which were people in the past. This handles all of the examples you list (leaving aside 3-year-olds, which are too close to the issue at hand), as well as explaining why I don't think interrupting compilation as above is killing a person but destroying the state of a running-but-paused AI does.

Questions for you:

• Does interrupting compilation as above seem to you like killing someone?
• If not, do you still think babies are closer to the examples you list than to this example?
• If not, do you still think babies are people?
• If so, can you think of some other example which we can both readily agree is a person (or not a person) which can help settle this?

Gametes are not people, even though we know how to make people from them, because the chance that any given sex cell ever becomes a person is so slim.

What's the cutoff probability?

Five months later...

What you're doing is referencing your feelings and seeing what the objects of those feelings have in common. [...] One of us just thinks they belong on the list and the other thinks they don't. [...] If we try to draw a line through the commonalities what are we going to get? [...] Conversely, what do all your other examples have in common that infants don't?

These all seem to indicate a bit of confusion about what I'm trying to do.

It seems to me that this thread of the debate has come down to "Should we consider babies to be people?" There are, broadly, two ways of settling this question: moving up the ladder of abstraction, or moving down. That is, we can answer this by attempting to define 'people' in terms of other, broader terms (this being the former case) or by defining 'people' via the listing of examples of things which we all agree are or are not people and then trying to decide by inspection in which category 'babies' belong.

Especially in light of the last line I've quoted above, you appear to be attempting the first method (and to be assuming I'm doing the same). In my experience, this method almost always ensures more confusion, not less, so I'm staying as far away from it as possible.

Instead I am attempting the second method. Here are several things I think are people: [adult] humans, strong AIs, thinking aliens. Here are several things I think are not people: pigs, chess-playing computer programs, rocks, dead humans. I am not going to attempt to suss the defining characteristics and commonalities of either category, because that would be an example of applying the first method and tends, as I say, to result only in more confusion.

Now, using only these categories (and in particular not using our feelings about whether or not babies are people), it seems to me that babies are less similar to the members of the first set than they are to the second. As such it seems that we ought to conclude babies are not people.

Worth repeating, I think: I am not going to attempt to list defining characteristics of the "people" and "not people" categories. Unless you can come up with what you think is a complete definition, that's not going to get us anywhere.

There are, at this point, several particulars on which you might disagree.

• You might think that this method of defining words is in some way invalid, especially in light of how difficult it would be to use this method with an AI's programming. [If this is the case, I challenge you to write a piece of software that, without appealing to you, identifies objects as people or non-people the same way you do it. Can't do this? Then we'd best stick with what tools we have available to us.]
• You might think that some of {humans, strong AIs, thinking aliens} are not people. [If this is the case, let's keep looking for a set we can agree on.]
• You might think that some of {pigs, chess-playing computer programs, rocks, dead humans} are people. [If this is the case, let's keep looking for a set we can agree on. Or you could announce that you're now, for moral reasons, vegetarian, in which case our moralities are likely irreconcilable without vastly more work.]
• You might think that babies are more similar to members of the first category than members of the second. [If this is the case, we're probably at an impasse. You or I might attempt to convince the other by expanding the sets I've outlined above with members we both agree belong until the similarities and differences are sufficiently stark - but frankly, at this point, it seems quite obvious that babies belong in the second category.]

A misc. point (the important part is above):

I strongly suspect that societies where people had no reluctance to go around offing their infants wouldn't have lasted very long.

This is entirely orthogonal to the point I was trying to make. Keep in mind, most societies invented misogyny pretty quick too. Rather, I doubt that you personally, raised in a society much like this one except without the taboo on killing infants, would have come to the conclusion that killing infants is a moral wrong.

There's a specific implication in your response about "the needs of society" to which I could respond but which I'm not going to unless prompted; I hope the above has dealt that.

I am keeping a log of my changes that I'm sure will be completely useless to everyone except me, but I'm tracking it anyway.

Well, I would be interested if you don't mind sharing such personal information. I want (to want) to embark on a similar journey and I could use some motivation.